Senior Data Engineer

About the Position

The Senior Data Engineer will be an integral part of the data management team, responsible for designing, building, and maintaining robust data pipelines and architectures that ensure the effective handling of large datasets within the banking domain. This role reports to the Data Engineering Manager and is critical for enhancing the organization's analytical capabilities.

Accountabilities

• Develop and optimize ETL pipelines to facilitate data ingestion and transformation from various sources into a centralized data lake.
• Utilize Databricks and PySpark to process large volumes of structured and unstructured data efficiently.
• Write complex SQL queries to perform data extraction, manipulation, and analysis to support business decision-making processes.
• Collaborate with cross-functional teams to identify data requirements and ensure that data solutions align with organizational goals.
• Implement best practices for data governance, quality assurance, and performance tuning within data processing workflows.
• Troubleshoot and resolve data-related issues in a timely manner to minimize disruptions in analytics operations.

Skills and Experience Required

• Extensive experience with Databricks, PySpark, Python, SQL, and ETL pipeline development.
• Proficiency in managing and architecting data lakes tailored specifically for analytical use cases.
• Strong understanding of the banking domain, including its data management challenges and regulatory requirements.
• Demonstrated ability to work on complex datasets with attention to detail while ensuring adherence to best practices in data engineering.
• Excellent problem-solving skills combined with a methodical approach to technical challenges.
• Strong communication skills to effectively convey technical concepts to both technical and non-technical stakeholders.