Engineering Manager - AL/ML

Department Icon Data Science Analytics & Machine Learning
Posted: 2 months ago
7-9 years
Gurugram, Haryana
Data Science Analytics & Machine Learning
work from office
Full Time

Posted: 2 months ago
This job's closing date (4 Nov 2026) has passed, so the role may already be filled. See live similar jobs.
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

Role Overview

Leena AIs agentic assistant answers employees HR, IT, and policy questions from each customers knowledge base, across 400 enterprise tenants.

This role leads the Knowledge Intelligence team: retrieval and ranking, answer grounding, conflict detection, LLM policy, and the evaluation infrastructure that keeps an LLM-powered agent trustworthy at enterprise scale.

The role blends people leadership, system design, and hands-on ownership of production LLM systems. The candidate we hire has shipped LLM-based products to real users — with evals, tool calling, and cost/latency engineering not only classical ML. Classical ML depth is necessary here; it is not sufficient.

LLM & Agentic Systems Ownership

  • Own production LLM systems end to end: RAG pipelines, agent tool-calling flows, model selection and routing, guardrails, and answer-vs-escalate decision logic.
  • Build and own evaluation infrastructure: golden datasets per tenant, LLM-as-judge graders calibrated against human judgment, regression gates on every prompt and model change, and online quality monitoring.
  • Drive answer quality: grounding and attribution against ACL-visible sources, hallucination control, and confidence calibration across heterogeneous tenant knowledge bases.
  • Engineer cost and latency at scale: routing to right-sized models, caching, and token budgets — with per-query economics you can state from memory.

Retrieval & ML Ownership
  • Own search quality across tenants: embeddings, rerankers, hybrid retrieval, chunking/sectioning, and measurement (recall@k, MRR) — including per-tenant diagnosis when quality regresses.
  • Guide the full model lifecycle: data preparation, training, evaluation, deployment, monitoring, and iteration — including the data-bias traps in learning from your own systems logs.
  • Ensure ML systems are reliable, observable, scalable, and cost-efficient in production.

Engineering Leadership
  • Lead, mentor, and grow a cross-functional team of backend and ML engineers.
  • Create a high-trust, high-ownership culture focused on quality, learning, and delivery.
  • Support career development, performance management, and technical growth of team members.

Architecture & Technical Direction
  • Drive architecture for distributed, multi-tenant backend systems that integrate LLM inference, retrieval, and data pipelines.
  • Separate what must be learned from what should be a rule: design systems where models handle judgment and policy handles constraints.
  • Partner with senior ICs on trade-offs across accuracy, latency, cost, and system complexity; review designs and code across services, ML pipelines, and infrastructure.

Execution & Cross-Functional Collaboration
  • Plan and execute multi-quarter roadmaps spanning product engineering and ML initiatives; balance experimentation velocity with enterprise-grade reliability and compliance.
  • Work closely with Product, Applied AI, and Customer teams to align technical execution with business outcomes; translate product requirements into feasible plans.
  • Represent engineering in roadmap, prioritization, and stakeholder discussions — with metrics, not adjectives.

RequirementsLLM Production Experience (hard Requirement)
  • Shipped LLM-based systems to production with real users — RAG, agents, or LLM-powered product features — and operated them: this is the bar, not a bonus.
  • Designed and run evaluation for LLM systems: offline golden sets, judge models validated against human labels, and eval-gated rollout of prompt or model changes.
  • Debugged real agent and tool-calling failures (malformed calls, loops, hallucinated grounding) and can tell those stories concretely, with numbers.
  • Reasoned about LLM system economics in production: cost per query, latency budgets, routing and caching strategies.

Experience & Background
  • 7+ years of software engineering experience, with prior hands-on IC work in ML or applied AI systems.
  • 2+ years managing engineers, ideally including ML or data-focused teams.
  • Experience building and operating production ML systems (not just experimentation or research).

Technical Expertise
  • Strong retrieval/NLP foundation: embeddings, rerankers, hybrid search, and search-quality measurement.
  • Strong foundation in distributed systems and backend engineering (Node.js or Python, APIs, cloud infrastructure); multi-tenant SaaS experience valued.
  • Ability to reason about ML-specific trade-offs: accuracy vs latency, offline vs online metrics, calibration, retraining strategies.

Leadership & Judgment
  • First-principles problem framing: interrogates objectives, cost asymmetries, and constraints before proposing architecture. Our interviews test this habit directly.
  • Proven ability to lead teams through ambiguity and technically complex problem spaces.
  • Comfortable being hands-on when needed, while primarily operating as a multiplier for the team.
  • Excellent communication with technical and non-technical stakeholders; can explain LLM behavior and risk to product and business partners.

How We Interview

We tell candidates this up front because it produces better conversations. Expect:

  • a deep-dive on an LLM system you personally shipped — architecture, eval story, failures, and cost;
  • a design problem from our domain, where asking the right questions is scored.

Education

Bachelors or Masters degree in Computer Science, Engineering, or a related field (Tier 1 preferred).

Nice to Have

  • Conversational AI, enterprise knowledge management, or search at multi-tenant scale.
  • Classical recommender/personalization systems experience (valued in addition to, not instead of, LLM production work).
  • Prior experience scaling AI systems in B2B SaaS or regulated, high-availability environments.

Skills: node.js,apis,machine learning,nlp,python,llm agents,engineering manager,generative ai,llm

Skills

PythonCloud InfrastructureMachine LearningAiMlLlmGenerative AiRagEmbeddingsTool CallingGuardrailsAgenticAgentic AiNlp

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Leena AI is a conversational AI platform that helps businesses automate their HR processes. Leena AI's AI-powered chatbot can handle a wide range of HR tasks, from answering employee questions to onboarding new hires. The platform is designed to be easy to use and integrate with existing HR systems.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Gurugram.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.