AI Quality Engineer

Department Icon Data Science Analytics & Machine Learning
Posted: 5 days ago
2-5 years
Hyderabad, India
Data Science Analytics & Machine Learning
work from office
Full Time
Closes 10 Jan 2027

Posted: 5 days ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

AI Summary
Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

AI summary of this job

Beta Read in 10 seconds

An AI Quality Engineer role at Deccanaiauto in Hyderabad for people with 2–5 years of experience. You'll design quality systems, analyze datasets, and build training programs using Python, SQL, LangChain, and LLM.

  • Hyderabad
  • Work from office
  • 2–5 years

What you'll do

  • Design and build quality systems
  • Analyze datasets for client teams
  • Turn findings into training content

Good fit if you

  • Have annotation industry experience
  • Bring strong Python and SQL skills
  • Demonstrate working ML and LLM knowledge
Sounds like you? Applying takes under a minute.

AI-generated from the job description. Check the full description before you apply.

Helpful?

Job Description

AI Quality Engineer (LLM Evals & Human Data)

About the role
Frontier models are now limited less by compute but more by  high quality human data and evaluations behind them. This role sits exactly at that intersection.
At Deccan AI, our annotators, raters and QCs produce evaluation and training data for some of the world's leading AI labs - multi-turn assistant evals, agentic tool-use tasks, A/B conversation evals, rubric-based rating and robotics video labeling. You will be the person who makes that data trustworthy at scale.
We are looking for an analytical thinker with high attention to detail and an uncompromising approach when it comes to data quality - along with the drive to deliver the best outcome for our customers (clients). You will need to have a strong backbone, often requiring pushing back on defective data outcomes.

What you'll own

  • Work with project teams to design and build quality systems, using a combination of agents and deterministic code, right from project’s kickoff to final delivery
  • Understand client guidelines, interpret nuanced requirements and design training programs to ensure effective talent deployment on projects.
  • Analyse datasets and findings to build insight reports for client research teams.
  • Turn quality findings into targeted feedback into example-driven training content on the LMS.

What you must bring

  • 2–3 years in the annotation industry - at a data vendor, an AI lab, or an in-house labelling team - working on evals, SFT, multimodal or agentic data.
  • Hands-on quality systems experience - you have designed or run QC workflows, sampling plans, rubrics or audit programmes.
  • Strong Python - you write scripts and small pipelines to process, score and analyse task data without using automations.
  • SQL fluency - comfortable joining messy production tables and building dashboards (Metabase, Superset, Looker or similar).
  • Working ML and LLM knowledge - how models are trained (SFT, RLHF, preference data), how evals work, and what inter-rater agreement, precision/recall and calibration mean in practice.
  • Strong analytical skills - you can break down large, messy datasets, spot patterns and root causes, and turn them into clear, defensible insights.
  • Ownership under pressure - you can run several projects at once on tight timelines and still hold the highest quality bar.

Great if you also have

  • Built agents or pipelines with LLM APIs, LangChain/LangGraph, or similar tooling.
  • Prepared golden-sets, calibrated LLM judges and hillclimbed judge performance.
  • Worked on agentic or tool-use evals, coding / SWE-bench-style tasks, or robotics and video labeling.
  • Experience with statistics for audits: confidence intervals, stratified sampling, Cohen's kappa / Krippendorff's alpha.

Why this role

  • Frontier-lab exposure from day one - your work goes straight into how leading AI models are trained and evaluated.
  • Build, don't maintain - many of our quality systems and auto-QC pipelines are being built now; you'll shape them, not inherit them.
  • Real autonomy - you own projects, setting quality across accounts.
  • The skill set the market is short of - ML + code + evals + human-data judgement is one of the hardest profiles to hire in AI; this role builds all four at once.

Skills

PythonSQLLangChainLLMStatisticsLookerCATeamsRobotics

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Deccan AI Auto is a company leveraging artificial intelligence and advanced technologies in the automotive sector.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Hyderabad.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.