Senior Azure Data Engineer

Role - Data Architect

Location - Remote

Responsibilities:

· Define and design scalable and secure architectures, frameworks and pipelines for building, deploying and diagnosing high performance data applications

· Collaborate with cross functional teams to build a comprehensive data platform

· Design and implement data storage solutions using Cosmos DB or graph databases, ensuring efficient data retrieval and storage

· Design and implement cloud solutions and build data pipelines on cloud solutions (e.g.,MS Azure)

· Implement caching strategies using technologies like Redis Cache to improve application performance

· Design API Orchestration/Scheduling using Airflow or similar

· Review data models, run code refactoring and optimization, containerization, deployment, versioning, and monitoring of its quality, including data & concept drift

· Provide test strategy and automation guidance for the transformed data platforms

· Provide best practices, guardrails and execution processes for automated and efficient data at scale

· Implement secure and reliable authorization mechanisms using industry-standard protocols

· Conduct performance analysis and optimization to identify and address performance bottlenecks

· Collaborate with DevOps teams to ensure smooth deployment, monitoring, and management of microservices

· Enable users & teams on the platform; troubleshoot and debug user issues; maintain user-friendly documentation

· Stay up to date with the latest trends and advancements in data science and cloud technologies

Requirements

· Must-have: Bachelors degree in Computer Science, Software Engineering, Mathematics or a related field. Masters degree is a plus

· Must-have: Minimum of 9+ years of hands-on experience including proficiency in object oriented programming languages like C/C++, Python, Java etc

· In-depth knowledge of data and services architecture and related concepts

· Must-have: Ability to design and implement on cloud platforms, preferably Azure

· Must-have: Proficiency in deploying applications using AKS (OR) Experience with NoSQL databases, specifically Cosmos DB or graph databases

· Preferred: Experience with authentication and authorization mechanisms such as OAuth 2.0 and OpenID Connect, Okta

· Preferred: Familiarity with containerization and orchestration tools like Docker and Kubernetes

· Proficiency in working with caching technologies like Redis Cache

· Strong understanding of software development best practices, including code reviews, testing, and version control

· Strong understanding of DevOps principles and practices, CI/CD, etc. and tools (Git, Azure DevOps, Airflow, etc)