Data Engineer

Department Icon IT / Software Development & Related
Posted: 1 year ago
5-7 years
Navi Mumbai, Mumbai
IT / Software Development & Related
Work from Office
Full Time

Posted: 1 year ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
Similar Jobs
Please verify your account first! Send OTP

Posted over 15 days ago, so the role may have closed. Confirm on the employer's page when you apply, or see similar jobs.

Job Description

ETL Migration Specialist (Informatica to Databricks)

Position Overview

We are looking for an experienced ETL Migration Specialist with strong expertise in Informatica PowerCenter and Databricks (PySpark). The ideal candidate will be responsible for analysing and reverse engineering existing Informatica mappings/workflows, understanding the business logic implemented, and refactoring these pipelines into scalable, efficient data processing workflows on Databricks.

This role requires strong analytical skills, hands-on experience in ETL development, and the ability to collaborate with both business and technical teams to ensure seamless migration.

Key Responsibilities

  • Reverse Engineering: Analyze existing Informatica mappings, workflows, sessions, and variables to understand data flow, transformations, joins, filters, lookups, and aggregations.
  • ETL Refactoring: Translate Informatica logic into PySpark/Databricks workflows, ensuring accuracy and performance.
  • Data Modeling & Transformation: Design and implement transformation logic in Databricks aligned with business rules.
  • Performance Optimization: Refactor ETL logic for scalability and performance tuning in Databricks.
  • Documentation: Create detailed documentation for the converted pipelines, capturing both technical logic and business rules.
  • Testing & Validation: Collaborate with QA/Business teams to validate data correctness between Informatica outputs and Databricks outputs.
  • Best Practices: Ensure adherence to Databricks and cloud data engineering best practices, including modular coding, error handling, and reusability.
  • Collaboration: Work with stakeholders, data architects, and business SMEs to clarify requirements and resolve discrepancies.

Must Have Informatica Expertise

  • Strong knowledge of Informatica objects:
    • Mappings, Mapplets, Workflows, Worklets, Sessions
  • Hands-on experience with Informatica transformations:
    • Lookup, Joiner, Router, Aggregator, Expression, Sequence Generator, Filter, Stored Procedure, Update Strategy, etc.
  • Proficiency in handling Informatica parameters and variables:
    • Mapping variables and parameters
    • Workflow variables
    • Session parameters
    • Parameter files and variable scoping
  • Ability to debug and trace variable values across sessions and workflows.
  • Experience in designing reusable transformations and optimization techniques in Informatica.

Required Skills & Experience

  • 5+ years of hands-on experience in ETL development with Informatica PowerCenter (or IICS).
  • 2+ years of experience in Databricks (PySpark, SQL, Delta Lake) for ETL/ELT implementation.
  • Strong understanding of data warehousing concepts (SCDs, fact/dimension modeling, partitioning, indexing).
  • Hands-on experience with cloud platforms (Azure / AWS / GCP) and Databricks workspace setup.
  • Proficiency in SQL and performance tuning of large datasets.
  • Experience in reverse engineering ETL logic and re-implementing in modern platforms.
  • Excellent problem-solving, debugging, and communication skills.

Good to Have

  • Experience with DevOps/CI-CD pipelines for Databricks (e.g., GitHub Actions, Azure DevOps, Jenkins).
  • Exposure to Informatica to Spark migration tools or accelerators.
  • Knowledge of Medallion architecture or modern data lakehouse patterns.
  • Experience in Agile/Scrum delivery.

Skills

MigrationInformaticaData BricksData MigrationETLCloud Data EngineeringPysparkBig DataDebuggingDevopsTestingCloudSql

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New IT / Software Development & Related roles in Navi Mumbai.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.