Databricks Data Engineer | Senior Data Engineer

• Databricks Data Engineer | Senior Data Engineer
• Company: talentmatics
• Job Location: Pune, Maharashtra, India
• Employment Type: Full-time
• Experience Required: 5+ years
• We are Hiring: Databricks Data Engineer | Senior Data Engineer | Migration / Technical Architect
• We are looking for skilled Databricks professionals to join our data engineering and migration teams across multiple experience levels.
• Experience : 5 + years
• Locations: Pune | Mumbai | Bengaluru | Delhi NCR
• Work Mode: Work From office
• Notice Period : Immediate to 30 days
• Key Responsibilities
• Assess legacy databases, ETL jobs, data warehouses, and reporting dependencies to identify migration, redesign, or retirement opportunities.
• Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, SQL, and Delta Lake.

Implement data ingestion and transformation using Unity Catalog, Lakeflow, Auto Loader, Databricks Workflows, and Databricks SQL.Migrate legacy data platforms and workloads from Oracle, SQL Server, SSIS, Informatica, Teradata, Snowflake, Hadoop, and other data platforms to Databricks.

• Optimize Spark and SQL workloads through performance tuning, partitioning, liquid clustering, Photon, and efficient compute utilization.
• Develop reusable, metadata-driven components to automate ingestion, conversion, validation, reconciliation, and deployment.
• Perform schema validation, record-count checks, business-rule testing, regression testing, parallel runs, and cutover readiness assessments.
• Implement production-ready logging, monitoring, restartability, security, and data quality controls.
• Follow CI/CD, DevOps, data governance, and deployment best practices.
• For senior and architect-level roles, lead migration workstreams, review solution designs, mentor engineers, and resolve complex technical challenges.
• Mandatory Skills
• Strong hands-on experience in Databricks, PySpark, Python, and SQL.Practical knowledge of Delta Lake, Unity Catalog, Lakeflow, Auto Loader, and Databricks Workflows.
• Experience in data engineering, ETL/ELT development, data modelling, and production data pipelines.
• Knowledge of incremental loads, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD).
• Hands-on experience with data validation, reconciliation, testing, and performance optimization.
• Understanding of CI/CD, workspace configuration, data governance, and production support.
• Ability to understand business requirements, data consumption patterns, and the software development lifecycle (SDLC). .