Senior Data Engineer

Senior Data Engineer - UAE
Position Overview
Location: Abu Dhabi, UAE
Work Requirement: Client requirement to have people onsite with office visits.
Key Responsibilities
The successful candidate will be responsible for:

• Implementing Databricks pipelines using Delta Live Tables (declarative pipelines).
• Rewriting proprietary Foundry transforms into optimised PySpark on Databricks.
• Moving from Foundry's build system to modern CI/CD (IaC-managed Databricks workspaces).
• Performing performance tuning on Databricks Photon to match or exceed Foundry interactivity.
• Contributing to the reconciliation engine (Palantir vs Databricks output equivalence).
• Managing pipeline rebuild, validation, internal testing, and UAT preparation.
  

Required Skills & Experience
Candidates must demonstrate strong hands-on experience in the following areas:

• Proven production-level Databricks experience.
• Strong hands-on coding capability.
• Experience building and optimizing modern data pipelines.
• Ability to work within pod-based delivery teams.
• Experience owning pipeline migration and rebuild activities from design through validation.
  

Databricks

• Databricks hands-on experience.
• Delta Live Tables.
• Unity Catalog.
• 
  Workflows.
• Production experience required (not lab-based experience).
  

Data Engineering & Development

• PySpark.
• Advanced SQL.
• Data modelling.
• Schema design.
• Medallion architecture.
• Lakehouse architecture.
  

DevOps & Deployment

• Modern CI/CD for data pipelines.
• Infrastructure as Code (IaC).
• Git-based deployment.
  

Communication

• Strong written and verbal English skills.
• Ability to interact directly with Databricks Professional Services (DBX).
  

Preferred Qualifications
The following experience is highly desirable:
Certifications

• Databricks Certified Data Engineer Professional.
• Associate certification at minimum.
  

Domain Experience

• Financial services.
• Audit.
• Asset management.
  

Platform Experience

• Prior Palantir Foundry exposure, including:
• Code Repo.
• Code Workbook.
• Pipeline Builder.

Migration Experience
Experience With Migrations From Legacy Platforms Such As

• Informatica.
• Hyperion.
• Oracle.
• Palantir Foundry.
• Databricks.
  

Skills
Candidates with any of the following experience will be advantageous:

• Azure Data Factory (ADF).
• Azure Synapse.
• dbt.
• Airflow.
• Equivalent orchestration tools.
  

Note: Client estate is Azure-heavy.