Lead Big Data DevOps Engineer

Department Icon IT / Software Development & Related
Posted: 1 year ago
4-8 years
Maharashtra, Pune
IT / Software Development & Related
Work from Office
Full Time

Posted: 1 year ago

This job is no longer accepting applications

It closed on 23 Oct 2025. Here are 6 live jobs like it - apply in one step.

See all live IT / Software Development & Related jobs
Job Description
Similar Jobs
Please verify your account first! Send OTP

Job Description

    As a Lead Big Data DevOps Engineer, you will be responsible for architecting, scaling, and optimizing Big Data infrastructure and DevOps practices. Your role will involve driving automation, reliability, and performance in complex, distributed data environments. Collaborating with cross-functional teams, you will enable fast, secure, and scalable data delivery. Additionally, as a technical leader, you will mentor engineers, establish best practices, and ensure the seamless integration of new technologies and processes.

    Your key responsibilities will include:

    Technical Leadership & Strategy:

    • Leading the design, implementation, and continuous improvement of Big Data infrastructure across on-premises and cloud environments.
    • Defining and enforcing DevOps best practices for automation, CI/CD, monitoring, security, and scalability in Big Data workflows.
    • Architecting and optimizing infrastructure for high availability, disaster recovery, and performance at scale.
    • Driving the adoption of infrastructure-as-code (IaC), containerization, and orchestration for reproducible and scalable deployments.
    • Evaluating, selecting, and integrating new tools and technologies to enhance platform capabilities.

    Automation & Operational Excellence:

    • Building and maintaining robust CI/CD pipelines for Big Data applications, data pipelines, and platform services.
    • Overseeing automated provisioning, configuration management, and orchestration.
    • Implementing and maintaining comprehensive monitoring, logging, and alerting solutions for proactive issue detection and resolution.
    • Ensuring end-to-end data pipeline reliability, including automated data quality checks, lineage, and observability.

    Collaboration & Mentorship:

    • Collaborating closely with data engineers, data scientists, and application teams to streamline workflows.
    • Mentoring and upskilling DevOps and engineering team members to foster a culture of knowledge sharing and continuous improvement.
    • Leading incident response, root cause analysis, and post-mortem processes for critical production issues.

    Security & Compliance:

    • Implementing and enforcing security best practices for Big Data environments, including RBAC/ABAC, secrets management, encryption, and compliance with standards.
    • Overseeing access controls, audit trails, and vulnerability management for all infrastructure components.

    Process & Governance:

    • Defining and documenting operational processes, disaster recovery/business continuity plans, and SLAs for platform services.
    • Supporting capacity planning, cost optimization, and resource management across clusters and environments.
    • Driving the adoption of DevOps and DataOps culture and practices across the organization.

    Required Qualifications:

    • Bachelors or Masters degree in Computer Science, Engineering, or related field.
    • 4-8 years of experience in DevOps, SRE, or infrastructure engineering roles, with at least 3 years leading Big Data or DevOps teams.
    • Advanced expertise in Big Data frameworks, containerization, orchestration, and infrastructure-as-code.
    • Strong scripting skills for automation and tooling.
    • Deep understanding of CI/CD pipelines, version control, and modern DevOps toolchains.
    • Experience with cloud platforms and security best practices.

    Preferred Qualifications:

    • Experience with advanced monitoring and observability tools.
    • Familiarity with data cataloging, metadata management, and data lineage tools.
    • Exposure to MLOps, GenAI platforms, or data mesh architectures.
    • Experience with NoSQL databases and distributed storage systems.
    • Prior experience in high-availability, mission-critical data environments.

    Benefits:

    • Competitive salary and benefits package.
    • Leadership role in building and scaling next-generation Big Data platforms.
    • Opportunity to work with cutting-edge technologies in a collaborative, high-impact environment.
    • Professional growth and development through challenging projects and continuous learning.

    If you are passionate about leading technical teams, driving automation and reliability at scale, and enabling data-driven innovation, join us as our Lead Big Data DevOps Engineer.,

Skills

DevopsNosql DatabasesVersion ControlCloudDevOps CultureSecrets ManagementOrchestrationAnsibleMonitoring SolutionsCloud PlatformsData Mesh ArchitecturesContainerizationKafkaScalabilityGitLab CIDockerSLAsCapacity PlanningDistributed Storage SystemsAutomationAirflowMonitoringAlertingPythonNoSQL DatabasesSparkResource ManagementFine TuningTerraformSecurityJenkinsComplianceDevOps PracticesCICD PipelinesMonitoring ToolsMongoDBOperational ProcessesObservabilityBusiness Continuity PlansObservability ToolsDisaster RecoveryDebuggingRBACABACData Lineage ToolsAWSGCPGitMetadata ManagementAzureMLOpsCost OptimizationBig Data InfrastructureInfrastructureascodeLoggingData CatalogingHadoopCICDDataOps CultureCassandraData Quality ChecksBashGenAI PlatformsEncryptionKubernetesSecurity Best PracticesTrino

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New IT / Software Development & Related roles in Maharashtra.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.