AI/ML Benchmark Analyst

At Crossing Hurdles, we work as a referral partner. We refer candidates to our partner that collaborates with worlds leading AI research labs to build and train cutting-edge AI models.

Position: AI/ML Benchmark Analyst

Type: Hourly Contract

Compensation: $25-$30 per hour

Location: India (Remote)

Commitment: 40 hours/week, flexible

Role Responsibilities:

• Review reinforcement learning (RL) environments and verify terminal conditions for correctness, consistency, and alignment with task objectives.
• Evaluate benchmarking pipelines to ensure accuracy, fairness, reproducibility, and rigorous experimental standards.
• Provide structured technical feedback on environment design, evaluation protocols, and implementation details.
• Review and analyze Python-based codebases to assess environment behavior, termination logic, and metric calculations.
• Collaborate with research and engineering teams to refine evaluation methodologies and benchmarking criteria.
• Validate reproducibility by checking performance across different runs, seeds, and hardware configurations.
• Document findings clearly and propose improvements to strengthen benchmarking reliability and system robustness.

Requirements:

• Background in reinforcement learning, computer science, or applied AI research.
• Experience working with RL environments and understanding how terminal conditions and environment dynamics are implemented.
• Strong understanding of benchmarking methodologies, evaluation metrics, and experimental protocols in RL.
• Proficient in Python; ability to review and reason about code (PyTorch/TensorFlow is a plus).
• Strong critical thinking and analytical skills for identifying inconsistencies and implementation issues.
• Detail-oriented, with a commitment to fairness, accuracy, and reproducibility in agentic AI research.

Key Domains: Candidates with experience in the following areas are encouraged to apply:

• Reinforcement Learning: Environment design, termination logic, reward structures.
• Benchmarking & Evaluation: Reproducibility testing, fairness assessments, metric development.
• Agentic AI Systems: Evaluation protocols for agentic reasoning and behavior.
• Python Code Review: Environment scripts, evaluation pipelines, simulation frameworks.
• Experimental Rigor: Seed management, cross-hardware validation, experimental stability.

Application Process:

• Apply for the job role.
• Await the official message/email from our recruitment team (typically within 1-2 days).