Data Engineer (Pyspark, Databricks, ADF)

Senior Data Engineer (PySpark, Databricks & ADF)
About The Opportunity
A fast-growing firm in the Data Engineering & Cloud Analytics sector delivering enterprise-grade data platforms and analytics solutions. We architect and operate scalable Lakehouse pipelines on Azure to drive reporting, ML, and real-time insights for global customers. This is an on-site role in India focused on building robust, production-grade ETL and data infrastructure using PySpark, Databricks and Azure Data Factory.
Role & Responsibilities

• Design and implement scalable batch and streaming ETL pipelines on Azure using PySpark and Databricks, ingesting data into Delta Lake.
• Author and maintain Azure Data Factory pipelines for orchestration, parameterization, scheduling, and operational runbooks.
• Tune and optimize Spark jobs for performance and costpartitioning, caching, shuffle optimization, and efficient file formats.
• Implement data quality, schema evolution, and lineage controls; build automated validation and reconciliation checks.
• Establish CI/CD for notebooks and infrastructure-as-code; manage source control, release pipelines, and environment promotion.
• Collaborate with data scientists, BI teams, and stakeholders to productionize models, expose curated datasets, and meet SLAs.
  

Skills & Qualifications
Must-Have

• PySpark
• Databricks
• Azure Data Factory
• Delta Lake
• Apache Spark
• Python
• SQL
• Git
  

Preferred

• Apache Airflow

Looking to get Placed? Try our Placement Guarantee Plan

Get Hired
• Azure DevOps
• Spark performance tuning
  

Qualifications

• Bachelors degree in Computer Science, Engineering or equivalent preferred.
• Proven experience building production data pipelines on Azure/Databricks; ability to work on-site in India.
  

Benefits & Culture Highlights

• Work on high-impact, enterprise-scale data platforms and Lakehouse architectures.
• Collaborative engineering culture with opportunities to mentor and define best practices.
• Exposure to modern analytics and MLOps workflows with rapid career growth potential.
  

Location: India (On-site) This role requires hands-on presence at an India office to collaborate with engineering and analytics teams.
Skills: adfs,databricks,apache spark,python,sql,pyspark,azure