Lead Data Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 2 years ago
3-5 years
Bengaluru, Karnataka, India
Data Science Analytics & Machine Learning
Work From Office
Full Time

Posted: 2 years ago

This job is no longer accepting applications

It closed on 30 Jan 2024. Here are 6 live jobs like it - apply in one step.

See all live Data Science Analytics & Machine Learning jobs
Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

Your Job

The Lead Data Engineer will be a part of global team that designs, develops, and delivers data engineering solutions leveraging the latest data engineering technologies for Koch Industries. Koch Industries is a privately held global organization with over 120,000 employees around the world, with subsidiaries involved in manufacturing, trading, and investments. Koch Technology Center (KTC) is being developed in India to extend its IT operations, as well as act as a hub for innovation in the IT function. As KGSI rapidly scales up its operations in India, its employees will get opportunities to carve out a career path for themselves within the organization. This role will have the opportunity to join on the ground floor and will play a critical part in helping build out the KGSI (Koch global services india) over the next several years. Working closely with global colleagues would provide significant global exposure to the employees.

Our Team

The Lead Data Engineer will report to the Data Engineering & BI Lead of the KGSI and will be responsible to develop and implement a future-state data analytics platform for both the back-end data processing and the front-end data visualization component for the Finance Data Delivery teams. This will be a hands-on role and the candidate will be responsible for designing & development of data frameworks.

What You Will Do

  • Be part of the data team to design and build a data engineering solutions.
  • Implement batch and near real time data movement design patterns and define best practices in data engineering.
  • Design and develop optimal cloud data solutions (lakes, warehouses, marts, analytics) by collaborating with diverse IT teams including business analysts, project managers, architects, and developers.
  • Work closely with a team of data architects, data engineers, BI developers and data analysts to procure, blend and analyze data for quality and distribution; ensuring key elements are harmonized and modeled for effective analytics, while operating in a fluid, rapidly changing data environment
  • Build data pipelines from a wide variety of sources

Demonstrate strong conceptual, analytical, and problem-solving skills and ability to articulate ideas and technical solutions effectively to external IT partners as well as internal data team members.Work with cross-functional teams, on-shore/off-shore, development/QA teams/Vendors in a matrixed environment for data delivery.Strong troubleshooting skills.Update and maintain key data cloud solution deliverables and diagram.Ensure conformance and compliance using KGSI architecture guidelines and enterprise data strategic vision

Who You Are (Basic Qualifications)

  • Bachelors degree in Computer Science, Engineering, or related IT area with at least 8 + years of experience in software development.
  • Primary Skill set: AWS cloud experience, SPARK, , PYTHON, Columnar DB(Redshift or Snowflake) and SQL, Any ETL tool Experience (Preference - Glue), Secondary Skill set : AWS IAM, STEP FUNCTIONS and Git, Databricks, Pyspark, Spark SQL
  • At least 5+ years of hands-on experience in designing, implementing, managing large-scale and ETL solutions.
  • At least 3 years of hands-on experience in data modelling, data engineering, ETL, multi-dimensional data warehouses, cubes, with expertise in relevant languages and frameworks like SQL, Python etc.
  • Hands on experience with designing and fine-tuning queries in Columnar DB(Redshift or Snowflake)
  • Understanding of common DevSecOps/DataOps and CICD processes, methodologies, and technologies like GitLab, Terraform etc
  • Be able to analyze large complex data sets to resolve data quality issues.

What Will Put You Ahead
  • AWS certifications like Solution Architect (SAA/SAP) or Data Engineering Associate (DEA)
  • Hands-on experience with AWS data technologies and at least one full life cycle project experience in building a data solution in AWS.
  • Exposure to visualization tools, such as Tableau or PowerBI.
  • Exposure with OLAP technologies and Data Virtualization (using Denodo)
  • Knowledge on manufacturing domain is preferable.

Other Consideration
  • May participate in a 24x7 on call rotation once the development is complete.
  • Shift Timings - 12PM to 9 PM

At Koch companies, we are entrepreneurs. This means we openly challenge the status quo, find new ways to create value and get rewarded for our individual contributions. Any compensation range provided for a role is an estimate determined by available market data. The actual amount may be higher or lower than the range provided considering each candidates knowledge, skills, abilities, and geographic location. If you have questions, please speak to your recruiter about the flexibility and detail of our compensation philosophy.

"Koch is proud to be a equal opportunity workplace."

Skills

AnalyticsData AnalyticsData ProcessingData QualityData VisualizationDesigningEtlEtl ToolFinanceInnovationOperationsProject ManagementPythonQualitySoftware DevelopmentSqlTableauVisualization

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Koch Global Services is the shared services organization of Koch Industries, one of the largest private companies in America. We deliver world-class capabilities and services to Koch companies, enabling them to be more competitive in their respective markets.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Bengaluru.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.