An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
Job Description
Role: QA Engineer Generative AI / Responsible AI Testing
Experience: 47 Years
Location: Open to any Orion Innovation Office in India ( Chennai/Pune/ Mumbai/Kochi/Coimbatore/Hyderabad)
Employment Type: FTE
Role Overview
We are seeking a detail-oriented QA Engineer with experience in Generative AI and Responsible AI testing to ensure the quality, safety, and reliability of AI-powered applications, including LLM-driven chatbots and intelligent systems. The role will focus on functional, adversarial, and ethical validation of GenAI solutions while supporting automation and performance testing initiatives.
Key Responsibilities
GenAI & Responsible AI Testing
- Design and execute prompt-based test cases covering functionality, bias, safety, and edge scenarios.
- Craft adversarial prompts to stress-test LLM behavior and identify vulnerabilities.
- Validate hallucination resistance, output consistency, and prompt reliability.
- Evaluate AI systems for bias, fairness, transparency, and ethical compliance.
- Test content moderation and safety filters for GenAI applications.
- Perform end-to-end QA for GenAI chatbot conversational flows.
Functional & System Testing
- Design test scenarios, test cases, and regression suites.
- Conduct System Integration Testing (SIT), Smoke Testing, Regression Testing, and Bug Testing.
- Execute manual and automated test cycles across releases.
- Maintain defect logs, triage defects, and track resolution.
Automation & Performance Testing
- Develop and maintain automation scripts using Python-based frameworks.
- Build automated tests using Pytest / Selenium / Robot Framework.
- Support load and performance testing to assess system scalability.
BI & Data Validation Testing
- Validate BI dashboards, reports, and visualizations.
- Ensure data accuracy, reporting reliability, and performance across tools like Tableau and Power BI.
- Support data storytelling and analytics validation.
Agile & DevOps Collaboration
- Participate in Sprint Planning, Backlog Grooming, and Daily Stand-ups.
- Collaborate with Developers, Product Owners, and Analysts to streamline QA processes.
- Support build deployments across QA, UAT, and Production environments.
- Provide feature demos and stakeholder updates.
Required Skills & Competencies
Testing Expertise
- Software Testing Life Cycle (STLC) & SDLC
- Test case design & execution
- Defect management & triaging
- Regression & integration testing
GenAI / AI Testing
- LLM testing & validation
- Prompt engineering for testing
- Responsible AI principles
- Bias & fairness evaluation
Automation & Tools
- Python
- Pytest
- Selenium / Robot Framework
- LangChain (preferred)
BI & Data Tools
- Tableau
- Power BI
- Advanced Excel
Project & QA Tools
- Jira
- Confluence
- Azure DevOps
Qualifications
- Bachelors Degree in Computer Science, Engineering, or related field.
- ISTQB CTFL certification preferred.
Nice to Have
- Knowledge of Machine Learning / Deep Learning concepts.
- Experience with GenAI governance frameworks.
- Exposure to statistical analysis and data science fundamentals.
Soft Skills
- Strong analytical & problem-solving skills
- Excellent communication & stakeholder management
- Attention to detail
- Ability to work in Agile teams
Skills
Data ValidationPythonData ScienceDeep LearningMachine LearningPrompt EngineeringAnalyticsAiIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in Pune.
One email a day. Unsubscribe any time.

