Software Dev Engineer Test I

AI Engineer (Software Developer in Test/QA)

Role Overview

An AI Testing Engineer / SDET will be responsible for strengthening software quality across Agile, cloud-native, and AI-enabled delivery teams. The role requires strong programming fundamentals, test automation expertise, and the ability to validate traditional applications as well as emerging Generative AI, RAG, and Agentic AI solutions. The engineer will work closely with software developers, architects, AI engineers, and DevOps teams to improve release quality, automate validation, reduce manual effort, and support AI-driven SDLC initiatives.

Key Responsibilities

Quality Engineering and Testing

• Understand application architecture, APIs, databases, source code, authentication flows, cloud integrations, and business workflows.
• Design, execute, and maintain test plans, test cases, and validation strategies across unit, functional, integration, system, API, end-to-end, and regression testing.
• Identify, document, track, and verify defects while ensuring the new functionality does not impact existing features.

• Develop and maintain reusable automation frameworks, test suites, validation scripts, and regression capabilities for UI, API, backend, and integration testing.
• Improve test coverage, release confidence, and engineering productivity by promoting automation-first quality practices.

• Design and execute evaluation strategies for Generative AI, LLM, RAG, and Agentic AI systems, including benchmark datasets and automated evaluation pipelines.
• Validate AI quality metrics such as accuracy, relevance, groundedness, faithfulness, hallucination detection, response quality, and latency.
• Test RAG workflows including ingestion, chunking, indexing, retrieval, ranking, response generation, prompt quality, agents, tools, workflows, and orchestration frameworks.

• 2 to 5 years of experience in software testing, quality engineering, test automation, reliability engineering, or similar roles.
• Bachelors degree in Computer Science, Information Technology, Engineering, or a related field.
• Strong programming skills in Java, TypeScript, JavaScript, or Python with good understanding of OOP, data structures, algorithms, and design patterns.
• Hands-on experience with automation tools such as Selenium, Playwright, Cypress, TestNG, Cucumber, RestAssured, PyTest, Postman/Newman, or similar frameworks.
• Practical experience in Agile and DevOps environments with Azure DevOps, Docker, Jenkins, Selenium Grid, TDD/BDD, and CI/CD practices.
• Exposure to frontend, backend, REST API, microservices, cloud-native applications, and database validation.
• Experience with Azure services such as App Services, Azure SQL Database, Azure Storage, Azure OpenAI, and Entra ID/OAuth is preferred.
• Knowledge of AI/LLM applications, RAG systems, LangChain, LangGraph, Semantic Kernel, Agentic AI frameworks, AI coding assistants, ML concepts, AI observability, vector databases, and performance tools such as JMeter or BlazeMeter will be a strong advantage.