Big Data Engineer

What role you will play in team: You will design, develop, and maintain big data infrastructure and pipelines to support various data science and analytics initiatives.

What you will do: You will work with large-scale data sets to build data pipelines, develop ETL processes, and ensure data quality and consistency.

Key responsibility:

• Design and build scalable and robust big data pipelines using technologies such as Spark, Hadoop, and Kafka.
• Develop and implement ETL processes for data ingestion, transformation, and loading.
• Ensure data quality, consistency, and integrity throughout the data pipeline.
• Monitor and optimize data pipeline performance.
• Collaborate with data scientists and engineers to meet their data needs.
• Stay updated with the latest advancements in big data technologies.

Required Qualification and Skills:

• Bachelor's or Master's degree in Computer Science or related field.
• Experience with big data technologies such as Hadoop, Spark, and Kafka.
• Proficiency in SQL and programming languages like Python or Java.
• Experience with cloud-based big data solutions (e.g., AWS EMR, Azure HDInsight).
• Strong problem-solving and analytical skills.

Benefits Included:

• Competitive salary and benefits package.
• Opportunities for professional development and training.
• Exposure to cutting-edge big data technologies.
• Collaborative and supportive team environment.

A Day in the Life: A typical day includes designing, building, and maintaining big data pipelines, troubleshooting data quality issues, collaborating with other engineers, and learning about the latest big data technologies.