GCP Data Engineer

Job Title: GCP Data Engineer

Experience: 6+ Years

Location: Navi Mumbai – Work From Office

Employment Type: Contract

Role Overview

We are looking for a hands-on GCP Data Engineer with strong expertise in Cloud Composer (Airflow/DAG Factory), Cloud Dataflow, BigQuery, Python, SQL, and Oracle-to-GCP data migration.

The ideal candidate will be responsible for developing, orchestrating, and supporting metadata-driven ingestion pipelines as part of an ongoing Data Warehouse Migration initiative. The role requires strong data engineering fundamentals, production support experience, and the ability to build reliable and scalable cloud data pipelines.

Key Responsibilities

• Develop, maintain, and optimize data ingestion pipelines using Google Cloud Composer, Apache Airflow, and DAG Factory.
• Design and implement scalable ingestion frameworks using Cloud Dataflow.
• Develop and manage historical and incremental data loading processes from Oracle to BigQuery.
• Build metadata-driven and reusable data pipeline frameworks.
• Design and maintain watermark, metadata, audit, and control tables in BigQuery.
• Implement data validation, reconciliation, and data quality checks across source and target systems.
• Monitor production pipelines and troubleshoot failures, performance issues, and data discrepancies.
• Support production deployments and resolve data pipeline incidents within defined SLAs.
• Optimize pipelines for performance, reliability, scalability, and cost.
• Follow established data engineering standards, coding practices, CI/CD, and deployment processes.
• Implement appropriate security controls using GCP IAM and least-privilege access principles.
• Collaborate with data architects, developers, business teams, and other stakeholders throughout the migration lifecycle.
• Create and maintain technical documentation for data pipelines, workflows, and operational processes.

• 6+ years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Composer and Apache Airflow.
• Strong experience developing Airflow DAGs using DAG Factory.
• Proven experience building ingestion frameworks using Google Cloud Dataflow.
• Strong proficiency in Python and SQL.
• Hands-on experience with Google BigQuery.
• Experience working with Google Cloud Storage (GCS) and Pub/Sub.
• Strong experience working with Oracle databases.
• Experience with Oracle-to-GCP data migration projects.
• Strong understanding of Data Warehousing, ETL/ELT, and incremental data loading.
• Experience implementing watermarking, metadata management, audit frameworks, and control tables.
• Strong understanding of data validation, reconciliation, and data quality practices.
• Experience monitoring, troubleshooting, and optimizing production-grade data pipelines.
• Good understanding of GCP IAM, security, and least-privilege access.
• Strong understanding of data engineering best practices and coding standards.

• Experience working on large-scale Data Warehouse Migration projects.
• Experience developing metadata-driven ingestion frameworks.
• Knowledge of GCP CI/CD and deployment methodologies.
• Experience with Cloud Monitoring and logging for production pipelines.
• Exposure to GCP networking and enterprise security concepts.
• Experience working in Agile/Scrum development environments.

Skills: pipelines,google,metadata,cloud,data,gcp,google cloud,data engineering