Data Engineer

Overview

We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines, data warehouses, and data processing systems. The ideal candidate should have strong skills in ETL development, cloud data services, and big data technologies.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines.
  
• Build and optimize data models, data lakes, and data warehouses.
  
• Develop scalable solutions for collecting, processing, and storing large datasets.
  
• Work with structured and unstructured data from various sources.
  
• Ensure data quality, consistency, and reliability across systems.
  
• Collaborate with data analysts, data scientists, and software teams.
  
• Automate data workflows and improve data pipeline performance.
  
• Implement best practices for data governance, security, and compliance.
  
• Monitor pipeline performance and troubleshoot issues.
  

Required Skills

• Strong programming knowledge in Python, Java, or Scala.
  
• Experience with SQL and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB).
  
• Hands-on experience with ETL tools and data pipeline frameworks.
  
• Experience with big data technologies (Hadoop, Spark, Hive, Kafka).
  
• Knowledge of cloud platforms such as:
  
• AWS (Glue, Redshift, S3, EMR)
  
• Azure (Data Factory, Synapse, Databricks)
  
• GCP (BigQuery, Dataflow, Dataproc)
  
• Familiarity with data warehousing concepts and dimensional modeling.
  
• Experience with version control (Git).
  
• Strong understanding of data structures, algorithms, and system design.
  

Preferred Qualifications (Optional)

• Experience with Airflow, Luigi, or similar orchestration tools.
  
• Experience with Docker, Kubernetes, and DevOps practices.
  
• Knowledge of Machine Learning pipelines (e.g., MLflow).
  
• Understanding of data security practices and GDPR.
  
• Experience with real-time processing using Kafka or Spark Streaming