Senior Data Engineer Kafka And Streaming Data Processing (Pune)

• Senior Data Engineer Kafka and Streaming Data Processing (Pune)
• Company: Nihilent
• Job Location: Pune, Maharashtra, India
• Employment Type: Full-time
• Experience Required: 10 - 12 years
• Senior Data Engineer Kafka & Streaming Data Processing
• Location: Pune
• Experience: 10-12 Years
• About the Role:

We're hiring a Senior Data Engineer to build and scale real-time data processing pipelines using Apache Kafka and Debezium on AWS and open source stack like Apache Iceberg, Open metadata, Atlas, Apache Airflow. If you're passionate about streaming architectures and enjoy working across the AWS data ecosystem and open source stack, this role is for you.

• Responsibilities:
• Design & manage Kafka-based streaming pipelines (topics, partitioning, consumer groups), Flink
• Design & manage Spark based Batch pipelines using Spark based framework
• Configure and maintain Debezium connectors for real-time CDC
• Build scalable ETL/ELT workflows using PySpark and AWS Airflow
• Strong experience in utilizing Iceberg tables and AWS open source system.
• Manage and optimize AWS Redshift for analytics & reporting
• Ensure pipeline reliability, data quality, and performance at scale
• Collaborate with cross-functional teams on data requirements
• What We're Looking For:
• 10-12 years of Data Engineering experience with strong hands-on Kafka, CDC, AWS skills (must-have)
• Experience with Debezium / CDC tools
• Proficiency in PySpark, SQL, AWS Airflow, Glue, Iceberg and Redshift
• Working knowledge of AWS services (S3, IAM, Lambda)
• Robust SQL and data modeling fundamentals
• Nice to Have:
• Experience in the Finance/Banking domain .