Junior Data Engineer

Role Overview

Key Responsibilities

• Design, build, and maintain ETL/ELT pipelines for large-scale data ingestion, transformation, and loading.
• Develop and optimize Spark and PySpark jobs for batch and real-time data processing.
• Work with AWS services (S3, Glue, Lambda, Redshift, Athena, EMR) to manage the data ecosystem.
• Support the design and implementation of Data Lake and Data Warehouse architectures.
• Implement data validation, partitioning, and schema management for efficient query performance.
• Collaborate with data analysts and BI teams to ensure data availability and consistency.
• Maintain data lineage, metadata, data quality, and governance.
• Implement monitoring and alerting for data pipelines.
• Use Git and CI/CD tools to manage code and automate deployment of data workflows.

Qualifications

• Education: Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• Experience: 3–5 years of hands-on experience in data engineering, pipeline development, or cloud-based data systems.
• Core Skills: Strong knowledge of SQL and Python/PySpark.
• Cloud & Data Stack: Practical experience with AWS (S3, Glue, Lambda, Redshift, Athena, EMR, Step Functions).
• Architecture: Solid understanding of Data Lake architecture, ETL/ELT frameworks, and data warehousing concepts.
• Frameworks: Familiarity with Delta Lake, Spark SQL, or other big data frameworks, along with data modeling and performance tuning.

Nice to Have

• Exposure to GCP (BigQuery, Dataflow) or Azure (Data Factory, Synapse, Databricks).
• Experience with Databricks, PostgreSQL, MySQL, NoSQL, or Airflow.
• Understanding of DevOps practices, CI/CD pipelines, and infrastructure automation.
• Prior experience in an EdTech or public data ecosystem.