Test Automation Engineer – AI Solutions

Department Icon Data Science Analytics & Machine Learning
Posted: 5 months ago
3-6 years
Bengaluru / Bangalore, Karnataka
Data Science Analytics & Machine Learning
work from office
Full Time

Posted: 5 months ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
Similar Jobs
Please verify your account first! Send OTP

Posted over 15 days ago, so the role may have closed. Confirm on the employer's page when you apply, or see similar jobs.

Job Description

Location: Bangalore / Mumbai

Role Overview

Seeking a QA Engineer with AI/ML testing experience to ensure the quality, reliability, and performance of AI-powered applications. The role involves validating AI models, LLM solutions, and data pipelines while identifying issues such as hallucinations, bias, model drift, and data quality concerns.

Key Responsibilities

  • Test AI/ML models, LLM applications, RAG systems, chatbots, and data pipelines.
  • Develop automated validation, regression, and performance testing frameworks.
  • Evaluate model outputs for accuracy, consistency, relevance, bias, and hallucinations.
  • Monitor production models for drift and performance degradation.
  • Validate data quality and conduct fairness, bias, and explainability assessments.

Must-Have Skills

  • Bachelors/Masters degree in Computer Science, Engineering, or related field.
  • 3 6 years of QA experience, including 1 2 years in AI/ML testing.
  • Strong Python skills for automation and data analysis.
  • Experience with RAGAS, DeepEval, Promptfoo, LangSmith, PyTest, Selenium, and Postman.
  • Good understanding of the ML lifecycle, data quality validation, and pipeline testing.
  • Strong analytical, communication, documentation, and stakeholder management skills.

Good-to-Have Skills

  • Prompt Engineering and LLM Red Teaming.
  • MLOps tools: MLflow, SageMaker, Vertex AI.
  • Vector Databases, Responsible AI, AI Safety, and Fairness Frameworks.
  • A/B Testing, Shadow Deployments, CI/CD, and DevOps practices.

Skills

PythonData AnalysisAi/mlPrompt EngineeringAiMl

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Bengaluru / Bangalore.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.