Data Engineer Lead

Key Responsibilities:

• Collaborate with business stakeholders to understand key performance indicators (KPIs) and their impact on business objectives.
• Translate business questions and KPIs into analytical frameworks and data requirements.
• Apply advanced analytical techniques to uncover underlying drivers and correlations within KPI data.
• Develop insightful visualisations and reports that effectively communicate KPI trends and patterns.
• Provide actionable recommendations to improve KPI performance and achieve business goals.
• Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
• Perform ETL (extraction, transformation, loading) from diverse data sources.
• Ensure data quality, consistency, and integrity through rigorous data cleaning and validation.
• Optimize data storage and retrieval for efficient analysis and reporting.
• Conduct exploratory data analysis (EDA) to identify patterns, trends, and anomalies.
• Apply statistical techniques (e.g., hypothesis testing, regression analysis) to validate findings and quantify relationships.
• Develop and implement predictive and prescriptive models using machine learning algorithms.
• Evaluate model performance and iterate to improve accuracy and robustness.
• Build and deploy machine learning models using frameworks like scikit-learn, TensorFlow, or PyTorch.
• Implement AI solutions to address specific business challenges and optimize KPI performance.
• Monitor and maintain deployed models, ensuring optimal performance and scalability.
• Apply MLOps best practices for continuous integration and deployment.
• Translate complex data insights into clear, concise, and actionable recommendations.
• Create compelling visualisations and dashboards to communicate findings effectively.
• Present data-driven insights to both technical and non-technical audiences.
• Document all code, models, and analytical processes for transparency and reproducibility.

Required Skills:

• 5+ years of hands-on experience in data analytics or business intelligence roles.
• Proficient in SQL (writing complex queries, joins, subqueries, CTEs, window functions).
• Strong expertise in data modeling, and report/dashboard creation.
• Experience working with Databricks or other modern data platforms (e.g., Snowflake, BigQuery, Redshift).
• Working knowledge of Python for data manipulation and visualisation.
• Understanding of data warehousing concepts, lakehouse architectures, and data integration.
• Excellent problem-solving, communication, and stakeholder management skills.

In this role, you will have the opportunity to work in a mid-size and fast-growing company, engage in exciting projects involving cutting-edge technologies, and benefit from professional development opportunities. The company promotes a friendly working environment without a cliche office hierarchy system, allowing you to contribute new ideas and be appreciated for your innovative thinking.,