Principal Data Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 1 week ago
8-15 years
Rai Durg, Hyderabad, Telangana, India
Data Science Analytics & Machine Learning
work from office
Full Time
Closes 2 Jan 2027

Posted: 1 week ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
Similar Jobs
Please verify your account first! Send OTP

Job Description

Job OverviewAt iCIMS, we're redefining how people connect with opportunity through intelligent, human-centered technology. We're growing rapidly and are seeking a highly experienced Data Engineering leader to build the next generation of our Talent Cloud platform through scalable data pipelines, modern data architectures, and analytics infrastructure that power data-driven decision-making and AI capabilities.This role requires deep expertise in Databricks and modern Lakehouse architectures, leveraging Apache Spark, Delta Lake, and cloud-native technologies to build scalable and reliable data platforms. You will lead the design, development, and optimization of enterprise data solutions while partnering with software engineers, data scientists, product leaders, and business stakeholders. This position combines hands-on technical leadership, architectural ownership, and team mentorship within a culture that values innovation, ownership, and continuous learning. Responsibilities Lead a team of data engineers, providing technical direction, mentorship, and performance management.Design, develop, and maintain scalable data pipelines to collect, process, and store data from multiple sources.Architect, build, and optimize Databricks Lakehouse solutions supporting enterprise analytics, reporting, AI/ML, and data products.Develop large-scale distributed data processing solutions using Databricks, Apache Spark (PySpark), and Delta Lake.Implement and manage Medallion Architecture (Bronze, Silver, Gold) data layers to improve data quality, governance, and usability.Build and optimize data infrastructure to support analytics, reporting, and AI/ML workloads at scale.Implement event sourcing and streaming architectures using platforms such as Kafka and AWS Kinesis for real-time data processing.Leverage Databricks capabilities such as Auto Loader, Delta Live Tables, Databricks Workflows, Unity Catalog, and MLflow to improve platform efficiency and governance.Establish best practices for Databricks workspace administration, cluster management, performance tuning, monitoring, and cost optimization.Apply data governance, security principles, and compliance frameworks to ensure data quality and regulatory adherence.Collaborate with data scientists, software engineers, analytics teams, and product leaders to deliver scalable and reliable data solutions.Drive the adoption of modern data engineering practices, frameworks, and tooling across the organization.Troubleshoot and resolve complex data-related issues while maintaining data quality, availability, and integrity.Contribute to architectural decisions and technology roadmaps for the organization's data platform.Mentor data engineers and provide technical leadership across multiple initiatives and teams. Qualifications Bachelor's degree in Computer Science, Engineering, Data Science, or related field (or equivalent professional experience).7+ years of experience building enterprise-scale data pipelines, platforms, and distributed data systems.Expert-level experience with Databricks and modern Lakehouse architectures.Deep expertise in Apache Spark, preferably PySpark, for large-scale distributed data processing.Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and enterprise Databricks deployments.Proficiency in Python and strong familiarity with Java.Advanced SQL skills and experience with relational and non-relational databases such as SQL Server, PostgreSQL, MySQL, and MongoDB.Strong expertise with cloud platforms, preferably AWS, including services such as S3, Redshift, IAM, and cloud-native data architectures.Experience designing and implementing modern data lakes, data warehouses, and Lakehouse solutions.Expertise with streaming platforms such as Kafka and AWS Kinesis and event-driven architectures.Strong understanding of data modeling, warehousing, schema design, and data lifecycle management.Experience with data transformation tools such as dbt and business intelligence platforms such as Looker.Experience building and exposing APIs for data consumption and integration.Strong understanding of Git, CI/CD pipelines, infrastructure automation, and software engineering best practices.Experience implementing enterprise data governance, security, and compliance controls across modern data platforms.Experience optimizing Databricks clusters, Spark workloads, storage formats, and platform costs.Strong analytical, troubleshooting, and problem-solving skills with a passion for continuous learning and innovation.Excellent communication and collaboration skills across technical and non-technical audiences.Demonstrated experience mentoring engineers, leading technical projects, and influencing architectural decisions.

Skills

PythonJavaSQLData ScienceData EngineeringLookerApache SparkPySparkKafkaDbtDatabricksRedshiftData ModelingMySQLPostgreSQL

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Rai Durg.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.