Senior Data Engineer
The Bank of New York Mel...- 1 month ago
- Hyderabad, Telangana, India
- Full Time
Please verify your account first! Send OTP
Job Overview
Functional Area
Data
Work preferred
Work from Office
Experience
Min Experience
3 Years
Max Experience
5 Years
Description
Your opportunity
- Build an end to end data engineering pipeline (data ingestion, transformation, and data modeling), both batch and real-time streaming and distributed processing back-ends
- Incorporate best practices in the data pipelines for high performance, fault tolerance, monitoring, and alerting data quality checks.
- Work on Data Warehouse system architecture and ingesting/pipelining of data, and tools to automate and orchestrate in delivering performance, reliability and operational efficiency
- Work closely with Analytic Engineers and Data Analysts for the collection/analysis of raw data for models that empower end users.
- Build and scale our warehouse platform for data ingest, logging, search, aggregation, viewing, and analysis
- Routinely write efficient, legible, and well-commented Python
- Clear interpersonal skills to deliver on complex, technical topic
- Apply principles of SDLC in data engineering workflows (CICD, testing, code reusability, monitoring & alerting)
- Work closely with upstream data teams to define and implement best practices in data instrumentation
- Bachelor’s degree or equivalent in computer science, information systems, or combination of education and related experience required
- 4+ years of professional experience working and developing using Python
- 4+ years of professional experience working with Python scripting experience (Unix, bash, python)
- Strong in writing complex SQL queries
- Minimum 3 years of experience with cloud data warehouses (Redshift, Snowflake, etc.)
- Attention to detail and consistency of implementation
- Experience with modern data technologies
- Knowledge/Experience working with Kubernetes
- Experience with automation and orchestration tools
Skills
AutomationData QualityPythonQualityRecruitmentSnowflakeSqlTransformation