Lead Data Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 5 hours ago
8-15 years
Delhi, India
Data Science Analytics & Machine Learning
work from office
Full Time
Closes 15 Jan 2027

Posted: 5 hours ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

AI Summary
Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

AI summary of this job

Beta Read in 10 seconds

A Lead Data Engineer role at Client of Weekday AI in Delhi for people with 8–15 years of experience. You'll build large-scale data pipelines using Python, Java, Scala, SQL, Apache Spark, Hadoop, Kafka, Airflow, Snowflake, BigQuery, Redshift, AWS, Azure, and Docker.

  • Delhi
  • Work from office
  • 8–15 years

What you'll do

  • Design robust batch and real-time data pipelines
  • Establish data governance and automated monitoring
  • Mentor and coach junior data engineers

Good fit if you

  • Have 6+ years of experience in data engineering
  • Know distributed computing frameworks and messaging systems
  • Show leadership aptitude in growing engineering teams
Sounds like you? Applying takes under a minute.

AI-generated from the job description. Check the full description before you apply.

Helpful?

Job Description

  • 𝗧𝗵𝗶𝘀 𝗿𝗼𝗹𝗲 𝗶𝘀 𝗳𝗼𝗿 𝗼𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗪𝗲𝗲𝗸𝗱𝗮𝘆'𝘀 𝗰𝗹𝗶𝗲𝗻𝘁𝘀
  • 𝗦𝗮𝗹𝗮𝗿𝘆 𝗿𝗮𝗻𝗴𝗲: 𝗥𝘀 𝟰𝟬𝟬𝟬𝟬𝟬𝟬 - 𝗥𝘀 𝟲𝟬𝟬𝟬𝟬𝟬𝟬 (𝗶𝗲 𝗜𝗡𝗥 𝟰𝟬-𝟲𝟬 𝗟𝗣𝗔)
  • Experience: 6+ yrs
  • Location: NCR, Delhi, India

As a Lead Data Engineer, you will spearhead the design, development, and optimization of large-scale data pipelines that support both batch and real‑time data processing initiatives across the organization. You will collaborate closely with data scientists, product managers, and infrastructure teams to ensure data integrity, scalability, and high performance at every stage of the pipeline, while driving continuous improvement through automation, monitoring, and best practices. Your leadership will shape the data architecture, protect data quality, and enable timely, actionable insights that inform business decisions. In a fast‑moving environment, you will balance technical excellence with strategic vision, fostering a culture of collaboration and innovation within the data engineering team.

Requirements

𝗞𝗲𝘆 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀

  • Design and implement robust, end‑to‑end batch data pipelines using tools such as Apache Spark, Hadoop, and Airflow.
  • Develop real‑time streaming pipelines with technologies like Apache Kafka, Flink, or Spark Structured Streaming to deliver low‑latency analytics.
  • Optimize pipeline performance through partitioning, indexing, and efficient data format selection (e.g., Parquet, ORC).
  • Establish data governance, quality checks, and automated monitoring to ensure data reliability and compliance.
  • Lead capacity planning and scaling strategies for data infrastructure, including cluster management and cost optimization.
  • Mentor and coach junior data engineers, fostering a collaborative and growth‑oriented team culture.
  • Collaborate with Data Scientists to translate analytical requirements into scalable pipeline solutions.
  • Drive integration of data pipelines with downstream applications, dashboards, and reporting tools.
  • Stay current with emerging data engineering technologies and evaluate their applicability to business needs.
  • Participate in incident response and root cause analysis to quickly resolve data pipeline failures.

𝗪𝗵𝗮𝘁 𝗠𝗮𝗸𝗲𝘀 𝗬𝗼𝘂 𝗮 𝗚𝗿𝗲𝗮𝘁 𝗙𝗶𝘁

  • 6+ years of experience in data engineering with a proven track record in both batch and real‑time processing.
  • Deep knowledge of distributed computing frameworks such as Hadoop, Spark, and Flink.
  • Proficiency in message queuing and stream processing systems like Kafka, Pulsar, or RabbitMQ.
  • Strong scripting skills in Python, Scala, or Java, and experience with SQL on large datasets.
  • Hands‑on experience with cloud data services (AWS Glue, Redshift, BigQuery, Snowflake, or Azure Data Factory).
  • Familiarity with containerization (Docker, Kubernetes) and CI/CD pipelines for data workloads.
  • Demonstrated ability to architect and scale data pipelines to handle petabyte‑scale volumes.
  • Excellent communication skills, capable of translating technical concepts for non‑technical stakeholders.
  • A proactive problem‑solver with a passion for data quality, governance, and automation.
  • Leadership aptitude, with experience building, mentoring, and growing data engineering teams.

Skills

PythonJavaScalaSQLData EngineeringApache SparkHadoopKafkaAirflowSnowflakeBigQueryRedshiftAWSAzureDocker

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Weekday is an AI-powered work assistant that helps automate HR workflows. We transform HR processes to empower the employees and managers of tomorrow.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Delhi.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.