Senior Data Engineer

• Employment Type: Full Time
• Location: Bangalore (Work from office 3 days/week)
• Role: Senior Data Engineer
• Experience: 8+ Years (Preferred)
• Key Skills: Snowflake | Databricks | SQL | Spark / PySpark

About the Role

We are looking for an experienced Senior Data Engineer to join our client, Ness Technologies, in Bangalore. This role involves designing and building modern, scalable data solutions using Snowflake, Databricks, and PySpark, with a strong focus on end-to-end data architecture, performance optimization, and governance.

Key Responsibilities

• 1. Design and implement end-to-end ETL/ELT pipelines using Databricks and Snowflake for structured, semi-structured, and unstructured data.
• 2. Build and manage scalable, secure data platforms using PostgreSQL and DynamoDB.
• 3. Develop real-time and batch ingestion pipelines from diverse sources (APIs, logs, files, databases).
• 4. Apply complex data transformations for analytics, reporting, and machine learning use cases.
• 5. Optimize pipeline performance and manage storage costs using best practices in partitioning and indexing.
• 6. Implement data quality checks, access controls, backups, and governance policies in line with enterprise standards.
• 7. Contribute to data strategy, unification, and architecture for large-scale platform integration.
• 8. Define data architecture principles, standards, and reusable patterns for project-wide consistency.
• 9. Design and oversee data lakes, warehouses, and mesh architectures.
• Provide technical leadership to data engineering teams in building scalable ingestion and transformation frameworks.

Core Skills

• 1. Proven expertise in building batch and streaming data pipelines using Databricks (PySpark) and Snowflake.
• 2. Strong programming knowledge in SQL, Python, and PySpark for high-performance data processing.
• 3. Hands-on experience with Azure and AWS data ecosystems, including Microsoft Fabric.
• 4. Proficient in processing structured (CSV, SQL), semi-structured (JSON, XML), and unstructured (PDF, logs) data.
• 5. Experience with low-latency, high-throughput pipeline design using Spark and cloud-native tools.
• 6. Familiar with CI/CD, automation, schema versioning, and data security in production systems.
• Interested one share your resume at nandini@elabsinfotech.com