Senior Quality & AI Evaluation Engineer (AIRE)

Department Icon Corporate Finance
Posted: 3 days ago
5-10 years
Pune OR Bengaluru, Multiple, India
Corporate Finance
work from office
Full Time
Closes 12 Jan 2027

Posted: 3 days ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

Role

Senior Quality & AI Evaluation Engineer (Senior)

Visit our website bmwtechworks.in to know more.

Follow us on LinkedIn | Instagram | Facebook | X for the exciting updates.

About the UNIT/ Unit Overview 

AIRE (AI-native Requirements Engineering) is a new BMW Group tool for managing vehicle requirements, the descriptions of what a car and its software must do. It keeps the full history of every requirement, shows how requirements are linked to each other and to tests, and shows what is affected when something changes. It helps teams meet automotive standards such as Automotive SPICE and ISO 26262. AIRE is built for AI agents first and a user interface later, and the team uses AI agents to help build it. In this role you make sure AIRE and its AI features work correctly, through automated tests and checks.

Location

Pune or Bangalore

Experience

Number of openings

1

What awaits you/ Job Profile 

  • • Define and maintain the test and evaluation strategy, including test levels, quality gates and evidence.
  • • Automate BDD scenarios from the acceptance criteria, contract tests and property-based tests.
  • • Run load tests against the quality goals and report the results.
  • • Build and run evaluations for agents and AI functions together with the Lead Agentic Engineering & AI Platform Engineer.
  • • Produce test evidence for tool qualification together with the Functional Safety & Process Compliance Specialist.
  • • Act as the human approver for changes to test oracles, evaluation datasets and quality-gate thresholds.

• Set up automated quality gates (tests, coverage, static analysis, performance and security tests) from the very beginning, and keep them mandatory for every change.

What should you bring along

  • • Degree in Computer Science or a related field, or equivalent relevant experience.
  • • Minimum of five years of test or quality engineering experience.
  • • Demonstrated experience automating tests for data-intensive systems.

Must have technical skill 

  • • Strong test engineering experience in Python (for example pytest), including test automation in CI/CD.
  • • Experience with BDD (for example Gherkin, behave or pytest-bdd) and contract testing.
  • • Experience with property-based testing (for example Hypothesis) for complex data logic.
  • • Experience with load and performance testing (for example k6 or Locust) against defined service-level objectives.
  • • Experience evaluating LLM-based systems and agents with golden datasets, regression evaluations and acceptance thresholds.
  • • Proficient AI-native engineering practice, including using agents to generate tests while keeping test oracles and evaluation criteria under human control.

Good to have technical skills

  • • Knowledge of ISO 26262 Part 8 tool qualification and of test evidence for Automotive SPICE.
  • • Experience with mutation testing and test-quality metrics.
  • • Experience with observability-based testing (for example OpenTelemetry).
  • • ISTQB certification.

Skills

PythonLLMCI/CDPyTestBDDPerformance TestingComplianceTeams

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

BMW TechWorks India is a joint venture and a 100% digital hub of the BMW Group, focusing on software development, digital transformation, and IT solutions for the global automotive giant.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Corporate Finance roles in Pune OR Bengaluru.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.