Junior Spark And Apache Kafka Big Data Analytics Engineer

What role will you play in the team

You will serve as an integral member of our core data engineering division, architecting high-throughput data streams and distributed processing systems. Your core mission is to ingest, transform, and store massive volumes of enterprise streaming data to fuel our real-time machine learning inference and business intelligence dashboards.

What you will do

You will build, maintain, and scale distributed computing clusters using Apache Spark and Apache Kafka. You will write efficient ETL pipelines to clean incoming telemetry, integrate disparate relational and non-relational databases, and ensure uncompromising data integrity across all downstream analytics applications.

Key responsibility

• Design, deploy, and scale real-time streaming pipelines using Apache Kafka and Apache Spark streaming APIs.
• Develop robust ETL and ELT data workflows utilizing Python, SQL, and distributed processing frameworks.
• Manage and optimize database schemas across PostgreSQL, MySQL, and MongoDB for high-concurrency access.
• Implement big data processing jobs on Hadoop clusters and cloud-based data lakes efficiently.
• Collaborate with data scientists and machine learning engineers to supply clean features for predictive models.

Required Qualification and Skills

• Bachelor’s or Master’s degree in Information Technology, Software Engineering, or Applied Statistics.
• 0-1 years of practical experience with big data technologies, specifically Apache Spark, Kafka, and Hadoop.
• Strong programming foundation in Python and structured query languages for database management.
• Familiarity with data visualization tools such as Tableau, Power BI, and Google Data Studio.
• Understanding of ETL tools like Talend, Informatica, and Apache Nifi for data orchestration.

Benefits Included

• Competitive base annual compensation supplemented by performance-linked quarterly bonuses.
• Comprehensive health insurance plan covering hospitalization, outpatient care, and wellness check-ups.
• Continuous learning stipend for acquiring advanced cloud certifications and big data credentials.
• Structured mentorship program pairing junior engineers with seasoned technical architects.
• Inclusive, collaborative office culture equipped with modern amenities, catering, and recreational areas.

A Day in the Life

You start your morning by analyzing Kafka topic lag and Spark cluster resource utilization metrics. Mid-morning involves debugging an ETL pipeline failure in an Apache Nifi workflow and optimizing a sluggish SQL query. After lunch, you write unit tests for a new streaming consumer application and participate in a sprint planning session with the wider data engineering squad.