Data Engineer

[Project Details]: This project is to design and develop data delivery on Sony Music Publishing’s data warehouse on AWS. [Technology and Sub-technology]: AWS [Base Location]: Bengaluru [Type]: Hybrid [Qualifications]: BE/B.Tech in Computer Science 4+ years experience. [Job Overview]: The Data Engineer is responsible for designing and developing data delivery on our data warehouse on AWS.This data will be consumed by visual dashboards/reports that Sony Music Publishing teams use to better understand trends and insights to improve market share/songwriter deals [Primary Skills]: Experience in data architecture including data modeling, data mining and data ingestion Experience with AWS associated technologies (S3 buckets, Glue, Data Pipeline, DMS, RDS, Redshift, Aurora, Lambda) Knowledge of creating ETL scripts with languages such as Python, Node.js, SQL Experience in data warehousing and big data Experience with Relational databases (SQL Server) Experience working in Agile/Scrum teams AWS Services/Skills Competency (Basic, Intermediary, Advanced) 1 Python Intermediary 2 PySpark Intermediary 3 EMR/Glue Advanced 4 CICD Intermediary 5 Serverless Framework Intermediary 6 Cloud Formation Templates Intermediary 7 Redshift Advanced 8 Lambdas Advanced 9 Step Functions Advanced 10 Cloud Watch Intermediary 11 ElasticSearch/Open Search Advanced 12 Kibana Advanced 13 Kinesis Advanced 14 Redshift Spectrum Advanced 15 DMS Advanced [Good to have Skills]: PySpark CICD Cloud Formation Templates [Responsibilities and Duties]: Works with product owners, developers and AWS Infrastructure team to design and develop ETL processes Ability to automate and optimize processes as much as possible Ability to work in an Agile/Scrum team Ability to problem solve and propose solutions Use established Analytics standards/processes in ETL processes Ability to communicate with technical and business teams Ability to learn new technology quickly [Keywords] Python PySpark EMR/Glue CICD Serverless Framework Cloud Formation Templates Redshift Lambdas Step Functions Cloud Watch ElasticSearch/Open Search Kibana Kinesis Redshift Spectrum DMS