Cloud Data Engineer

Experience: 4-6 yrs

Key Responsibilities

1 Design and develop Engineering solutions to aggregate and automate large scale data flows from varying sources

2 Build ETLs from internal and external sources to provide insights into the business

3 Writing and tuning complex Data Pipelines, workflows, and data structures

4 Help continually improve ongoing reporting and analysis processes, automating or simplifying operational support for stakeholders

5 Create and maintain workflow/technical documents for all the development activities

6 Experience in communicating with business stakeholders as well as with colleagues

Technical Experience

1 Experience in industry big data platform stack like Databricks, Spark, Hadoop, Kafka, AWS DMS, AWS Fargate etc.

2 Must have skills Spark, Kafka, Airflow, Python, Glue, functional programming,

3 Deliver right solution architecture, automation and technology choices starting from experimentation and proof of concept phases of new analytical models

4 Ability to design logical and physical data models, big data storage architecture, data modelling methodologies, metadata management AWS Serverless native services

5 Strong experience on Data warehouse technologies, ETL, SQL Server