An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
AI summary of this job
Beta Read in 10 secondsAn AI Quality Engineer role at Deccanaiauto in Hyderabad for people with 2–5 years of experience. You'll design quality systems, analyze datasets, and build training programs using Python, SQL, LangChain, and LLM.
- Hyderabad
- Work from office
- 2–5 years
What you'll do
- Design and build quality systems
- Analyze datasets for client teams
- Turn findings into training content
Good fit if you
- Have annotation industry experience
- Bring strong Python and SQL skills
- Demonstrate working ML and LLM knowledge
AI-generated from the job description. Check the full description before you apply.
Job Description
AI Quality Engineer (LLM Evals & Human Data)
What you'll own
- Work with project teams to design and build quality systems, using a combination of agents and deterministic code, right from project’s kickoff to final delivery
- Understand client guidelines, interpret nuanced requirements and design training programs to ensure effective talent deployment on projects.
- Analyse datasets and findings to build insight reports for client research teams.
- Turn quality findings into targeted feedback into example-driven training content on the LMS.
What you must bring
- 2–3 years in the annotation industry - at a data vendor, an AI lab, or an in-house labelling team - working on evals, SFT, multimodal or agentic data.
- Hands-on quality systems experience - you have designed or run QC workflows, sampling plans, rubrics or audit programmes.
- Strong Python - you write scripts and small pipelines to process, score and analyse task data without using automations.
- SQL fluency - comfortable joining messy production tables and building dashboards (Metabase, Superset, Looker or similar).
- Working ML and LLM knowledge - how models are trained (SFT, RLHF, preference data), how evals work, and what inter-rater agreement, precision/recall and calibration mean in practice.
- Strong analytical skills - you can break down large, messy datasets, spot patterns and root causes, and turn them into clear, defensible insights.
- Ownership under pressure - you can run several projects at once on tight timelines and still hold the highest quality bar.
Great if you also have
- Built agents or pipelines with LLM APIs, LangChain/LangGraph, or similar tooling.
- Prepared golden-sets, calibrated LLM judges and hillclimbed judge performance.
- Worked on agentic or tool-use evals, coding / SWE-bench-style tasks, or robotics and video labeling.
- Experience with statistics for audits: confidence intervals, stratified sampling, Cohen's kappa / Krippendorff's alpha.
Why this role
- Frontier-lab exposure from day one - your work goes straight into how leading AI models are trained and evaluated.
- Build, don't maintain - many of our quality systems and auto-QC pipelines are being built now; you'll shape them, not inherit them.
- Real autonomy - you own projects, setting quality across accounts.
- The skill set the market is short of - ML + code + evals + human-data judgement is one of the hardest profiles to hire in AI; this role builds all four at once.
Skills
PythonSQLLangChainLLMStatisticsLookerCATeamsRoboticsIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in Hyderabad.
One email a day. Unsubscribe any time.

