Senior Data Engineer

We are seeking a highly skilled Senior Data Engineer to join our dynamic team in Karnataka or Tamil Nadu. The ideal candidate will have extensive experience in big data technologies and a strong background in data engineering. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and architectures that support our data analytics and business intelligence initiatives.

Key Responsibilities:

• Design and implement robust data pipelines using Hadoop, PySpark, and Airflow to ensure efficient data processing and integration.
• Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets for analysis.
• Optimize SQL queries and Hive scripts for performance and scalability.
• Monitor and troubleshoot data pipeline performance, ensuring data quality and reliability.
• Work with cross-functional teams to gather requirements and translate them into technical specifications.
• Stay updated with the latest industry trends and technologies in data engineering and big data.

Required Skills:

• Proficiency in Hadoop ecosystem components and frameworks.
• Strong programming skills in Python, with experience in PySpark.
• Solid understanding of SQL and experience with Hive.
• Familiarity with workflow orchestration tools like Airflow.
• Experience working in a Unix/Linux environment.
• Excellent problem-solving skills and attention to detail.
• Strong communication skills and ability to work collaboratively in a team environment.

If you are passionate about data engineering and looking for an opportunity to make a significant impact, we encourage you to apply for this exciting role.