Data Engineer

Job description / Role

About the role

We are seeking a skilled data engineer to join our growing team in Abu Dhabi. In this role, you will design, build, and maintain robust, scalable data pipelines and platforms that support analytics and reporting. You will work closely with data scientists, analysts, and stakeholders to ensure data accessibility, quality, and performance across our Azure-based ecosystem.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Apache NiFi and Apache Airflow.
• Implement real-time streaming data solutions with Apache Kafka, Spark Structured Streaming, and Apache Flink.
• Ingest data via change data capture (CDC) tools and manage data formats such as JSON, Avro, and Parquet.
• Build and optimize data lake architectures on Azure Data Lake Storage and Databricks, including Delta Lake implementations.
• Develop and enforce data modeling standards and best practices for structured and semi-structured data.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Monitor, troubleshoot, and optimize data pipelines for reliability, performance, and cost efficiency.
• Ensure data quality, governance, and security standards are followed throughout the data lifecycle.

Required qualifications

• Bachelor’s degree in computer science, engineering, or a related field.
• 4–7 years of professional data engineering experience.
• Proficiency in Python and advanced SQL query writing.
• Hands-on experience with Apache NiFi for data ingestion.
• Strong expertise in Apache Kafka, Spark, Flink, and Airflow.
• Experience implementing CDC solutions and working with common data formats, including JSON, Avro, and Parquet.
• In-depth knowledge of Azure services, including Data Lake Storage, Databricks, and Delta Lake.
• Solid understanding of data modeling principles and best practices.
• Experience with version control systems such as Git.
• Excellent problem-solving, communication, and collaboration skills.

Preferred qualifications

• Azure Data Engineer Associate or equivalent certification.
• Experience with containerization (Docker) and orchestration (Kubernetes).
• Familiarity with CI/CD pipelines for data workflows.
• Knowledge of data governance, cataloging, and security frameworks.
• Experience with machine learning pipelines or MLOps.
• Domain experience in finance, healthcare, or telecommunications.

Position details

Location: Abu Dhabi, United Arab Emirates

Experience required: 4+ years

About the Company

AIQU connects future-ready technology specialists with forward-looking organisations. We help businesses deliver high-impact and high-quality technology projects by using our unrivalled global sourcing capabilities to attract diverse talent within the region.

Our bespoke staffing solution offers the most flexible option you can have while maintaining 100% legislative compliance. AIQU’s fast and highly responsive employee management processes are designed to provide your most valuable assets, your people, with a seamless experience!

From business analysts to change managers; from front-end developers to back-end engineers, full-stack developers, and database administrators; from process analysts to software architects; AIQU can provide and manage specialists for a technology transformation project or for a project in maintenance phase across six key verticals.