Lead Data Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 3 weeks ago
8-15 years
Bangalore, Hybrid, India
Data Science Analytics & Machine Learning
work from office
Full Time
Closes 21 Dec 2026

Posted: 3 weeks ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Posted over 15 days ago, so the role may have closed. Confirm on the employer's page when you apply, or see similar jobs.

Job Description

About Arcana

Arcana was recently named one of LinkedIn's Top Startups of 2026, ranking among 50 rising companies in the U.S. alongside Anthropic, Perplexity, Cognition, and others. Arcana is a front-office portfolio intelligence platform built for hedge funds and asset managers. We empower investment professionals—from CIOs and portfolio managers to analysts and risk teams—with the tools to understand performance and risk, construct portfolios, evaluate investments, and make better investment decisions. The institutional investment software stack hasn't kept up with how modern investment teams work, and we're rebuilding it from the ground up.

The Role

As a Lead Data Engineer, you will take ownership of designing and delivering high-performance data platforms that enable analytics, AI/ML, and real-time insights across the organisation. You will mentor junior engineers, drive architectural decisions, and collaborate with stakeholders to align the data platform roadmap with business objectives. This is a hands-on technical leadership role for someone who enjoys both coding and guiding a team toward innovative solutions.

Responsibilities:

  • Lead the design and implementation of scalable and reliable data pipelines using Distributed Computing, Kafka, Airflow and Temporal.
  • Architect data solutions that support business analytics, and real-time processing.
  • Mentor and guide junior engineers, conducting code reviews and setting coding standards.
  • Collaborate with cross-functional teams to identify and solve data challenges.
  • Drive improvements in infrastructure, automation, and monitoring to enhance scalability and performance.
  • Ensure data quality, governance, and compliance with internal policies and industry best practices.
  • Stay current with the latest data trends and technologies, and make recommendations to keep the platform ahead of the curve.

Requirements:

  • 6-8 years of experience in data engineering or distributed systems.
  • Strong hands on programming skills in Python / Scala / Rust / Golang, and SQL.
  • Proven experience with cloud-native services (AWS, GCP) and big data tools (e.g., Spark, Flink, Datafusion, Doris, Trino etc).
  • Expertise in building real-time pipelines using Kafka, and batch pipelines.
  • Experience working with CI/CD pipelines, Kubernetes, Docker, and IaC tools.
  • Demonstrated ability to lead and mentor teams in a fast-paced, agile environment.
  • Strong problem-solving and communication skills with the ability to collaborate with multiple stakeholders.
  • Contributions to open-source projects or experience in a fast pace product-based company

What We Offer:

  • Competitive compensation and benefits package.
  • Opportunity to work with modern technologies and drive innovation.
  • Flexible working hours with Hybrid options.
  • A collaborative environment focused on learning and growth.

Skills

PythonRustScalaSQLData EngineeringKafkaAirflowAWSGCPDockerKubernetesCI/CDComplianceAgileCommunication

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Arcana Analytics is a data analytics and healthcare intelligence company providing advanced data-driven solutions.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Bangalore.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.