Machine Learning Researcher

Role: ML Researcher

Function: Machine Learning Research / AI Evaluation

Location: Hyderabad, India

Type: Full-time

Industry: Information Technology & Services, Computer Software, Artificial Intelligence

About Company

The company is a Series A GenAI data startup headquartered in Mountain View, CA. It runs the post-training and production layer for the worlds leading AI labs and enterprises.

Founded in 2024, it has grown rapidly — 10x last year — and serves a majority of the Magnificent 7 and top frontier labs globally. Its global network of 1M+ domain experts powers proprietary platforms including Helix (a hybrid eval suite) and EnterpriseOS (a scaled ops automation platform).

Backed by A91 Partners, Susquehanna International Group, and Prosus Ventures, with clients including Google DeepMind and Snowflake.

Position Overview

This role sits at the core of the companys research function, responsible for advancing how frontier AI systems are measured and understood. The ML Researcher will design novel benchmarks and evaluation datasets that go beyond existing industry standards, translating cutting-edge academic research into actionable frameworks used by the worlds top AI labs. The work directly influences how leading models are trained, tested, and improved at scale.

Role & Responsibilities

• Continuously track AI research publications, conferences, and emerging model architectures to identify gaps and opportunities in existing evaluation practices
• Design novel AI benchmarks and evaluation datasets across coding and non-coding domains, defining metrics that provide actionable insights into model capabilities and limitations
• Produce high-quality benchmark proposal documents covering problem definition, design structure, evaluation metrics, novelty rationale, and testing guidelines
• Translate insights from the latest AI papers into practical benchmark and dataset designs aligned with organisational and client priorities
• Collaborate with the ML Lead, pipeline-focused MLEs, and project managers to ensure research outputs are implementation-ready and meet project goals
• Iterate on existing benchmarks and datasets based on post-implementation feedback, new research findings, and evolving client use cases
• Deliver monthly research summaries on emerging AI trends impacting evaluation methodology and benchmark design

Must Have Criteria

• Proven experience designing AI benchmarks or evaluation datasets from the ground up, covering benchmark structure and metric definition
• Deep expertise in at least one of: agent frameworks, deep learning, NLP, or computer vision — with working knowledge of evaluation methodology in that domain
• Demonstrated ability to conduct in-depth literature reviews and translate academic research into practical, implementable frameworks
• Experience assessing and critiquing existing industry benchmarks for relevance, coverage gaps, and methodological limitations
• Strong technical documentation skills — prior experience producing requirement documents, research reports, or whitepapers that are implementation-ready

Nice to Have

• Familiarity with reinforcement learning environments or RLHF pipelines and their evaluation challenges
• Prior work at or in collaboration with AI labs, frontier model teams, or AI data/evaluation companies
• Published research or conference contributions in AI evaluation, benchmarking, or related ML domains
• Experience designing datasets for LLM post-training, fine-tuning, or model alignment

What We Offer

• Direct impact on how the worlds leading AI labs measure and improve frontier models
• Collaboration with a research and engineering team working at the cutting edge of LLM post-training and evaluation infrastructure
• High-growth environment backed by leading investors with significant runway and momentum
• Access to a global network of 1M+ domain experts and proprietary evaluation platforms (Helix and EnterpriseOS)