Senior Databricks Engineer

Department Icon IT / Software Development & Related
Posted: 1 week ago
5-10 years
Hyderabad, India
IT / Software Development & Related
work from office
Full Time
Closes 4 Jan 2027

Posted: 1 week ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
Similar Jobs
Please verify your account first! Send OTP

Job Description

About Aubrant 

Aubrant Digital is a leader in multi-shore custom application development. We are passionate about solving our clients’ business problems through consultative teamwork, innovative software, and proven processes. We’ve served more than 50 clients and delivered hundreds of high quality, custom enterprise applications.  Our clients value us as integral team members who get the job done on time and on spec, and we are proud of our high client retention rate and under 2% staff turnover. With offices in New Jersey, Boston, Costa Rica, and Eastern Europe, we execute the full software lifecycle, from architecture and design through development, QA and application maintenance & support.  Our company culture emphasizes client service, trust-based relationships, and innovation. 

About the role

The Senior Databricks Engineer will design, build, and optimize scalable data ingestion and transformation pipelines to support enterprise‑grade data modernization programs. This role focuses on implementing ELT workflows within modern Data Lakehouse architectures, ensuring high‑quality, reliable, secure, and efficient movement of data from diverse source systems into curated analytical and application‑ready layers.

You will work closely with Data Architects, Data Modelers, Cloud Engineers, and Application teams to develop repeatable processing frameworks, enforce data quality rules, and operationalize Medallion‑layered datasets (Bronze/Silver/Gold or equivalents). This role is domain‑agnostic but requires the ability to translate complex business and technical requirements into performant data pipelines. Experience with high‑volume operational data domains (e.g., logistics, telematics, IoT, ERP, mobility platforms) is a plus.

Key Qualifications:

  • • Bachelor’s/Master’s in Computer Science, Information Systems, Engineering, or related field (or equivalent experience).
  • • 5–8 years of experience building data ingestion and transformation pipelines.
  • • Strong proficiency in SQL and PySpark for large‑scale data processing.

• Hands‑on experience with dbt (data build tool) for building, testing, and documenting modular data transformation pipelines — a core requirement for this role.

  • • Hands‑on experience with Databricks, Spark jobs, and Delta Lake‑based transformations.
  • • Experience with ELT patterns and Medallion Architectures (Bronze/Silver/Gold).
  • • Practical knowledge of Airbyte, ADF, or equivalent orchestration tools.
  • • Experience with data quality frameworks such as Great Expectations or Databricks expectations.
  • • Strong understanding of schema evolution, partitioning, optimization, and Delta Lake performance tuning.
  • • Familiarity with Azure components including ADLS Gen2, Event Hub, Synapse Serverless/Dedicated SQL.
  • • Experience integrating pipelines with applications and APIs (read/write patterns).
  • • Strong debugging skills for data integrity, ingestion failures, schema drift, and performance issues.
  • • Excellent communication, documentation, and cross‑team collaboration skills.

Responsibilities:

  • • Build scalable ELT pipelines using Databricks (PySpark/SQL), dbt, and Delta Lake.
  • • Develop Bronze, Silver, and Gold transformations aligned with enterprise modeling standards.
  • • Implement ingestion using tools like Airbyte, ADF, or custom connectors.
  • • Collaborate with Data Modelers to operationalize business rules, mappings, and canonical definitions.
  • • Integrate data pipelines with downstream applications (mobile apps, web apps, APIs).
  • • Implement data quality and validation rules, error handling, and automated remediation steps.
  • • Manage schema evolution, CDC ingestion, late‑arriving data, and incremental refresh patterns.
  • • Optimize Spark/Delta workloads through partitioning, indexing, caching, Z‑ordering, and cluster tuning.
  • • Develop reusable pipeline templates and frameworks for cross‑domain scalability.
  • • Collaborate with Cloud Engineers to ensure pipelines meet security, monitoring, and DevOps standards.
  • • Produce technical documentation, lineage, and metadata for governance teams.
  • • Support ML workloads by preparing feature datasets and inference‑ready tables.
  • • Participate in cross‑functional design sessions, reviews, and architectural decisions.
  • About Aubrant & Our Culture

Skills

SQLPySparkDbtDatabricksAzureServerlessDevOpsQALogisticsCommunicationTeamsIoT

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New IT / Software Development & Related roles in Hyderabad.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.