Azure Data Engineer (Mid-Level)

Key Responsibilities:

• Data Pipeline Development: Design, build, and maintain robust, scalable, and efficient data pipelines using Azure Data Factory, Databricks, and other Azure services.

• Data Integration: Integrate structured and unstructured data from diverse sources into Azure-based data platforms.

• Data Modeling: Develop and maintain data models to support analytical and reporting requirements.

• Optimization: Monitor and optimize data pipelines for performance, scalability, and cost-efficiency.

• Database Management: Administer and manage Azure Synapse Analytics, Azure SQL Database, and other database solutions.

• ETL Processes: Design and implement Extract, Transform, Load (ETL) processes to transform raw data into actionable insights.

• Collaboration: Work closely with data analysts, business teams, and other stakeholders to understand data needs and deliver effective solutions.

• Security and Compliance: Ensure data solutions align with organizational security policies and industry regulations.

• Documentation: Create and maintain comprehensive documentation for data workflows, architectures, and processes.

Qualifications:

Technical Skills:

• Strong proficiency in Azure Data Services, including Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and Azure Storage.

• Experience with big data technologies such as Spark and Hadoop.

• Proficiency in SQL for querying and managing relational databases.

• Familiarity with NoSQL databases, such as Cosmos DB or MongoDB.

• Strong understanding of data integration and transformation using ETL/ELT methodologies.

• Hands-on experience with programming languages such as Python, Scala, or Java.

• Knowledge of data visualization tools like Power BI is a plus.

• Experience with Data Lake architecture and design.

• Understanding of security and governance in Azure, including RBAC and Azure Key Vault.

Experience:

• 3 5 years of experience in data engineering or a similar role.

• Experience working with large datasets and optimizing data pipelines in cloud environments.

• Hands-on experience in implementing end-to-end data solutions on Azure.

Soft Skills:

• Excellent problem-solving and analytical skills.

• Strong written and verbal communication abilities.

• Team player with the ability to work collaboratively and independently.