Data Engineer

We are seeking a skilled Data Engineer to join our dynamic team in Andhra Pradesh. The ideal candidate will have a strong background in data engineering, with expertise in AWS, Apache Spark, and Python. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines that support our data analytics and business intelligence initiatives.

Key Responsibilities:

• Develop and maintain robust data pipelines using Apache Spark and PySpark to process large datasets efficiently.
• Utilize AWS services to manage and store data, ensuring high availability and security.
• Collaborate with data scientists and analysts to understand data requirements and deliver solutions that meet their needs.
• Implement data integration solutions using Apache Airflow for workflow management.
• Work with messaging systems like Kafka and MQTT to facilitate real-time data processing and streaming.
• Write efficient SQL queries to extract and manipulate data from various databases.
• Monitor and optimize data pipeline performance, ensuring data quality and reliability.

Required Skills:

• Proficiency in AWS Data Engineering tools and services.
• Strong experience with Apache Spark and PySpark.
• Solid programming skills in Python.
• Expertise in SQL for data manipulation and querying.
• Familiarity with Apache Airflow for workflow orchestration.
• Knowledge of Kafka and MQTT for data streaming.
• Excellent problem-solving skills and attention to detail.

If you are passionate about data engineering and eager to work in a collaborative environment, we encourage you to apply for this exciting opportunity.