Azure Databricks Platform Architect / Technical Lead

Hybrid preferred however can be Pan India Remote for the right candidate 

Azure Databricks Platform Architect / Technical Lead

About the Role

We are looking for an experienced Azure Databricks Platform Architect / Technical Lead with strong hands-on experience in enterprise data platform architecture, infrastructure, and Databricks development.

Responsibilities:

• Design and implement enterprise-scale Azure Databricks platform architecture, including multi-workspace environments and Azure landing zones.
• Provision and configure Databricks workspaces, clusters, cluster policies, and compute resources.
• Design Azure networking solutions, including VNets, subnets, NSGs, CIDR planning, IP allocation, and secure connectivity.
• Configure and manage Unity Catalog, including metastore setup, RBAC, permissions, data lineage, and governance.
• Implement platform security, including Azure Key Vault, secrets management, credential rotation, and auditing.
• Automate infrastructure provisioning and deployments using Terraform, Bicep, ARM Templates, and CI/CD pipelines.
• Define platform standards for monitoring, performance, cost optimization, and operational support.
• Support Databricks development, including PySpark, Spark SQL, Delta Lake, and Medallion architecture.
• Lead technical design decisions and collaborate with Data Engineering, DevOps, Infrastructure, and Security teams.
• 
  

Must have Requirements
 8+ years of experience in Data Engineering, Cloud Infrastructure, or Platform Architecture, including at least 3 years with Azure Databricks.

The ideal candidate should be able to design, provision, and manage enterprise-scale Databricks environments, including networking, security, governance, and platform operations. Azure Databricks Networking, VPC management, IP management.

Must have worked and configured at least 1 enterprise level multi-tenant data platform.

 Hands-on experience designing and provisioning Azure Databricks workspaces from scratch.
 Strong experience with enterprise Databricks architecture, including multi-workspace environments and secure connectivity.
 Strong Azure networking knowledge, including VNets, subnets, NSGs, CIDR planning, routing, and IP address management.
 Hands-on experience with Unity Catalog metastore setup and configuration, RBAC, access controls, and data lineage implementation.
 Experience with Azure security, Key Vault, secret rotation, and credential lifecycle management.
 Experience with Infrastructure as Code, preferably Terraform, and CI/CD automation.
 Strong understanding of Databricks clusters, cluster policies, compute management, and performance optimization.
 Experience with enterprise data governance, monitoring, auditing, and production support.
 Good understanding of Python, PySpark, SQL, Delta Lake, and Medallion architecture.
 Experience leading technical design and architecture discussions.

 Experience with Databricks Asset Bundles (DABs).

Nice to Have
 Experience with multi-region Azure deployments and complex enterprise networking.
 Experience with Azure Monitor, Log Analytics, and cost governance.
 Experience in banking, insurance, or other regulated industries.
 Azure or Databricks certifications.