An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
Job Description
- Data Engineers Expert
- Company: KPIT
- Job Location: India
- Employment Type: Full-time
- Experience Required: 5 - 10 years
- We are hiring a Data Engineer (5+ Years Experience) for our heavy data analytics project.
- Location: Pune
- Experience: 5-10 Years
- Key Skills: Spark, PySpark, Scala, Python, SQL, Databricks, Data Lake, DWH, Snowflake, Azure ADF/Synapse/ADLS
We are looking for an experienced Data Engineer (5+ years) with strong handson expertise in building highperformance data pipelines for a heavy data analytics environment. The ideal candidate must excel in Spark (PySpark/Scala), complex SQL, data lake architectures, and data warehouse modeling. Experience with cloud data platforms such as Databricks, Azure, or Snowflake will be a strong advantage.
- Key Responsibilities
- Data Pipeline & ETL/ELT Development
- Develop and optimize Spark (PySpark/Scala) pipelines
- Ingest, transform, cleanse, and aggregate large datasets
- Build scalable batch and near realtime pipelines
- Apply Delta Lake optimization, partitioning, caching, and performance tuning Heavy Data Analytics
- Write complex aggregation logic (window functions, grouping sets, analytical functions)Understand KPIs, metrics, and analytical use cases
- Translate business logic into technical transformations
- Validate outputs against business requirements Data Lake Engineering
- Build multilayer data lake (Bronze/Silver/Gold)Work with Parquet, Delta Lake, ORC and columnar formats
- Implement schema evolution, metadata management, and auditing Data Warehouse Engineering
- Design dimensional models: Star Schema, Snowflake Schema
- Build fact and dimension tables
- Optimize table structures and partition strategies Databricks (Preferred)Develop notebooks and workflows in PySpark/Scala
- Manage clusters, jobs, and Delta Live Tables
- Apply best practices for cost and performance SQL Engineering
- Write and optimize complex SQL queries
- Perform data profiling, validation, and analytical computations
- Support dashboards and reporting layers Cloud Data Platforms
- Azure: ADF, Synapse, ADLS Gen2 (preferred)Snowflake: Warehouses, Snowpipe, Streams/Tasks
- Optional: Azure Functions Data Quality & Documentation
- Validate data transformations and business rules
- Document data flows, transformation logic, and metadata
- Collaborate with QA and analysts for data accuracy
- Required Qualifications
- 5+ years of hands-on experience in Data Engineering
- Strong programming: Spark, Scala, Python
- Strong SQL (complex joins, analytical functions, aggregations)Experience in Data Lake & Data Warehouse concepts
- Spark performance tuning (Delta, shuffle tuning, partitioning)Experience with at least one cloud ecosystem (Azure/AWS/GCP)
- Preferred Skills
- Experience with Databricks
- Snowflake or modern cloud DWHADF/Synapse/Airflow/dbtCI/CD for data pipelines
- Large-scale analytics environment exposure
- Soft Skills
- Strong understanding of business logic behind analytics
- Ability to convert metrics into technical transformations
- Excellent debugging & problemsolving capabilities
- Good communication and crossfunctional collaboration
- Required Skills
Spark, PySpark, Scala, Python, SQL (advanced), Data Lake, Data Warehouse, ETL/ELT pipelines, Delta Lake optimization, Azure/AWS/GCP cloud experience, Data modeling, Complex aggregations, Performance tuning
Supported Skills
Databricks, Snowflake, ADF, Synapse, Airflow, dbt, CI/CD for data pipelines, Largescale analytics e .
Skills
Data LakeDWHAzure ADFScalaSQLDatabricksPythonSparkPySparkSnowflakeIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in All India.
One email a day. Unsubscribe any time.

