Lead Big Data Engineer

Role Objective:

• Big Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data
• flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who
• enjoys optimizing data systems and building. The Data Engineer will support our software developers, database architects, data
• analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing
• projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products

Roles & Responsibilities:

• 1.Strong SQL Skills: Candidate should be proficient in writing and executing complex SQL queries (no PL/SQL) and be able
• to handle real-world scenarios confidently.
• 2. Hands-on Functional Programming (Scala): Strong hands-on experience in Scala is essential.
• Solid Data Engineering Fundamentals: Emphasis is on strong foundational understanding rather than tool-specific
• expertise.
• 3.Experience in Data Quality & Optimization: Candidate should have a sound understanding of data quality practices,
• performance optimization techniques, and solution design in both event-driven and batch processing contexts including
• data modeling.
• 4.Sound knowledge in Spark architecture and distributed computing and Spark streaming, including RDD and Data frames
• core functions, troubleshooting and performance tuning.
• 5.Experience in analyzing, re-architecting, and re-platforming on-premises data warehouses to data platforms on any cloud
• (AWS is preferred)
• 6. Experience in any cloud (ETL/ELT pipelines, Datalake and Data streaming) is preferred.
• 7. Understand the client requirement and able to design, develop from scratch and deliver.
• 8.. Able to mentor new members for onboarding to the project.

EXPERTISE AND QUALIFICATIONS

Experience: 5 to 9 years

Must have Skills:

• Scala/Java/React
• Spark
• SQL
• Python
• Any cloud (AWS preferable)
• Kafka /Kinesis/Any streaming services
• Hive, ETL/ELT design experience
• • CI/CD experience (ETL pipeline deployment)
• • Data Modeling experience
• • Data Warehouse experience (Snowflake)
• Good to Have Skills:
• • Git/similar version control tool
• • Knowledge in Microservices
• • Airflow or similar tools