Data Scientist / Lead Data Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 2 months ago
5-7 years
Bengaluru / Bangalore, Karnataka
Data Science Analytics & Machine Learning
work from office
Full Time

Posted: 2 months ago
This job's closing date (23 Oct 2026) has passed, so the role may already be filled. See live similar jobs.
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

Senior Data Engineer / Data Platform Lead

The Role

Every number a customer argues about.

Architecture is still open and yours to decide and defend: warehouse shape, table format, orchestration, where the streaming boundary sits against change capture or a tighter batch cadence, and how datasets are tiered with real service levels behind them.

This is not an Analytics Engineer role. They decide what a term means; you decide whether the number is computed correctly, arrives on time, and can bear the inference drawn from it.

  • If a metric means two different things in two places, that is theirs. If it means the right thing but is stale or irreproducible, it is yours.
  • What You Would Actually Be Doing
  • Own the Event Pipeline
  • Own the event pipeline behind shipment milestones: late arrivals, corrections that restate a milestone recorded three days ago, and the gap between event time and ingest time.
  • Define how a restatement propagates to something a customer has already seen.
  • Build for Agent-Grade Serving
  • Make the serving path good enough for an agent to depend on — with a stated freshness target per dataset and a read path whose p99 you know.
  • Because when a run stalls waiting for a number, a workflow with money attached stalls too.
  • Make Data Quality Fail, Not Warn
  • Build data quality as failing checks, not dashboards:
  • Null rates
  • Referential integrity breaks
  • Duplicate milestones
  • Distribution shifts when a partner changes a field
  • A check that only warns is a check nobody reads.
  • Enforce Tenant Isolation
  • Enforce tenant isolation at the data layer and assert it through tests.
  • Contract rates are commercially sensitive between competitors who may share the same forwarders.
  • Own Expensive Backfills
  • Plan backfills that cost real money.
  • A reprocess across 30+ countries of historical data has a bill and a blast radius. You own both.
  • Apply Statistical Judgement
  • Bring statistical rigour to decisions.
  • Know what you can and cannot claim from observational data — and say so before the decision, not after it.
  • What We Look For
  • Must-Haves

Years are a floor, not the bar.

If you miss one line but are strong on the rest, apply and tell us which one.

  • 5+ years building and operating production data platforms, including at least 2 years as the person accountable for architecture others depended on.
  • Designed a warehouse from first principles — grain, dimensional models, slowly changing dimensions, additive vs. non-additive measures — and can describe a modelling trap you found in someone elses work and fixed.
  • Deep SQL and Python: window functions, incremental and idempotent transformations, safe-to-rerun backfills, and pipelines that survive late-arriving data.
  • Hands-on ownership of a production orchestrator — Airflow, Dagster, Prefect, Temporal, or equivalent — including retries, service levels, dependency modelling, and backfill strategy.
  • A defensible position on table formats and streaming vs. change capture vs. tighter batch cadence, with the reasoning rather than the fashion.
  • Genuine statistical rigour: can design an experiment, defend the power calculation, and pick the right test for skewed or correlated data.
  • Has led engineers — hiring, reviews, mentoring — and carried a pager for datasets other teams depended on.

Also Good — None Required

  • You have run a data on-call rotation and can describe what you changed to make it quieter.
  • You have worked in a domain where being wrong about a time or quantity had a cost someone could name.
  • By Month Six
  • What Good Looks Like
  • We would rather tell you now what we would be measuring, so you can decide whether this is the job you want.

01 — Dataset Reliability

Every dataset a workflow depends on has a stated freshness target and a failing check behind it.

02 — Tenant Isolation

Tenant isolation in the data layer is asserted by a test that runs on every change.

03 — Team & On-Call Impact

You have hired or levelled up at least one engineer, and the data on-call is quieter than when you arrived.

Skills

PythonData EngineerAnalyticsSqlData Scientist

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

GoComet is a world-leading freight intelligence and supply chain visibility SaaS platform, helping global enterprises optimize their supply chains. We are built on a foundation of trust and security, and maintaining the integrity of our systems and customer data is paramount. We are looking for an experienced security leader to own our information security, compliance, customer trust, and governance programs while supervising organizational IT operations.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Bengaluru / Bangalore.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.