Senior Data Engineer

Position: Sr. Data Engineer

Work Location: Chennai

Experience : 5 to 10 Years

:

SUMMARY OF ESSENTIAL JOB FUNCTIONS

• Ensure deployment of modern data structures and models to enable reliable and scalable data products and feature stores
• 
  Develop and manage end-to-end data pipeline and application stack (Hadoop, Vertica, Tableau, Hue, Superset etc) and lead/provide end-to-end operation support for all applications.
• Develop processes for automating, testing, and deploying the work
• Identify risks and opportunities of potential logic and data issues within the data environment
• Perform RCA and resolution activities for complex data issues.
• Develop and maintain documentation relating to all assigned systems and projects
• Ability to independently perform proof of concepts for new tools and frameworks and present to leadership
• Work in an agile environment. Develop and drive multiple cross-departmental projects.
• Establish effective working relationships across disparate departments to deliver business results.
• Collaborate effectively with the global team and ensure day-to-day deliverables are met

REQUIREMENTS

• 5 to 10 years of Strong Experience with big data tools and data processing: Hadoop, Spark, Scala, Kafka, Yarn cluster, Java, etc.
• Strong development experience with EMR and SPARK is a must-have
• Strong experience with object-oriented/object function scripting languages: SQL, Python, Java, Scala, etc.
• Must have 5+ years of working experience with AWS
• Experience with GCP is nice to have [Note: GCP is a must past experience with AWS good as the customer is going from AWS to GCP]
• Experience with SQL/NoSQL databases like Vertica, Postgres, Cassandra, etc.
• Experience with data modeling concepts is desired
• Experience with streaming /event driven technologies work such as Lambda, Kinesis, Kafka, etc.
• Nice to have but not required - exposure to ML (frameworks like pytorch/tensorflow) and application development experience with Talend, Tableau, Snowflake, Redshift etc.