Lead Applied AI Platform engineer
Key Responsibilities
•Own end-to-end data load lifecycles across warehouse and lakehouse platforms, including ingestion, transformation, data quality validation, reconciliation, scheduling, and production support.
•Architect dimensional and lakehouse data models, along with the governance and access controls that go with them, including Unity Catalog.
•Design and build batch and streaming pipelines using Microsoft Fabric (Data Factory, Dataflows Gen2, notebooks, Lakehouse, Warehouse) and Databricks (Spark, Delta Lake, Workflows), selecting the right platform for each client context.
•Design and deliver GenAI solutions on enterprise data, such as RAG applications, vector search, and embedding pipelines, including the guardrails and evaluation approach.
•Own delivery from mobilisation through UAT, go-live, hypercare, and transition to operations, managing scope, budget, risks, and stakeholders.
•Track pipeline SLAs, data quality, cost consumption, and business value, and escalate early when delivery drifts.
•Lead, mentor, and grow a multidisciplinary team of data engineers, BI developers, and GenAI engineers, and contribute to hiring and capability building.
Required Skills and Attributes
•Bachelor’s or master’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related discipline.
•8+ years of technology experience, including significant experience delivering enterprise data programme’s and leading delivery or technical teams.
•Hands-on depth in Databricks, including Spark or PySpark, Delta Lake, Databricks Workflows, and Unity Catalog.
•Hands-on depth in Microsoft Fabric, including Lakehouse, Warehouse, OneLake, Data Factory pipelines, Notebooks, and Power BI integration.
•Proven experience owning end-to-end data load pipelines in a cloud environment.
•Strong dimensional modelling and data warehousing fundamentals, including star schemas, slowly changing dimensions, and performance optimisation.
•Practical experience delivering GenAI solutions on enterprise data, such as RAG, vector search, or LLM-based applications.
•Advanced SQL and strong Python skills, with working knowledge of CI/CD and cloud platforms (Azure, AWS, or Google Cloud).
•Strong communication, stakeholder management, and decision-making skills.
#pxe