Job Description
We are looking for an experienced QE?III (Senior Quality Engineer) to lead quality validation for our agentic AI models product to assess compliance, reasoning outcomes, and evaluation status.
This role requires strong expertise in test automation using Selenium WebDriver, CI/CD enablement, and solid experience validating AI-driven systems, including model outputs, confidence scores, and decision traceability. The QE?III will act as a quality owner, partnering closely with Product, Engineering, and Data Science teams.
Responsibilities
- AI & Compliance Quality Ownership
- Own end?to?end quality strategy for the AI agent models/ Agentic AI platform, covering UI, backend services, and AI model outputs.
- Validate AI agent behavior, including:
- Decision correctness
- Evaluation status
- Confidence thresholds
- Explainability / traceability of responses
- Define test approaches for non?deterministic AI behavior (golden datasets, tolerance-based assertions, confidence validation).
- Partner with Data Science teams to validate model inference results, regression drift, and version-to-version consistency.
- Design, develop, and maintain robust Selenium WebDriver automation frameworks (Java preferred).
- Implement automation for:
- Web UI workflows
- Compliance dashboards
- Agent evaluation flows
- Admin & configuration screens
- Ensure automation follows industry best practices: Page Object Model (POM), reusable utilities, proper wait strategies, and stable selectors.
- Page Object Model (POM)
- Data?driven and keyword?driven testing
- Reusable libraries and utilities
- Robust synchronization and wait strategies to reduce flakiness
- Integrate automation suites into CI/CD pipelines using:
- Maven
- Jenkins
- GitHub
- Own Jenkins pipelines for automated test execution (smoke, regression, AI evaluation suites).
- Enable automation execution as part of:
- Pull request validation
- Nightly regression
- Release readiness checks
- Analyze test failures, improve flakiness, and ensure high signal?to?noise ratio in automated results.
- Produce actionable test metrics (coverage, defect leakage, automation ROI).
- Drive functional, integration, and regression testing across multiple services and AI agents.
- Validate:
- API integrations (REST)
- Data flows across agent pipelines
- Evaluation status lifecycle (queued ? processed ? evaluated ? completed).
- Perform exploratory testing on new AI capabilities and edge cases.
- Act as QE SPOC for the pod / product area.
- Mentor junior engineers on automation, testing strategies, and AI validation approaches.
- Participate in requirement reviews and proactively identify quality risks early in the SDLC.
Looking to get Placed? Try our Placement Guarantee Plan
- 69+ years of experience in Software Quality Engineering.
- Strong hands?on experience in test automation using Selenium WebDriver.
- Proficiency in Java (or similar OOP language) for automation.
- Solid experience working with:
- Maven (build & dependency management)
- Jenkins (CI pipelines)
- GitHub (version control, PR workflows)
- Experience testing AI/ML?driven applications, analytics platforms, or rule/decision engines.
- Understanding of:
- Model inference pipelines
- Confidence scores
- Evaluation metrics
- Handling non?deterministic outputs
- Ability to design validation strategies for AI agents, beyond simple pass/fail testing.
- Strong experience with:
- API testing (REST)
- TestNG / JUnit
- Agile/Scrum methodologies
- Familiarity with test management and defect tracking tools (Azure DevOps TFS, etc.).
- Experience validating complex workflows across UI + API layers.
- Experience in compliance, audit, or regulatory systems.
- Exposure to performance testing (e.g., JMeter) for dashboards or AI pipelines.
- Knowledge of logging, monitoring, and observability tools for troubleshooting AI behavior.
- Experience validating data privacy and security aspects of AI systems.
Skills
Data PrivacyData ScienceDefect TrackingScrumSelenium WebdriverAi/mlAnalyticsAiMlIf an employer asks you to pay any kind of fee, please notify us immediately. Jobaaj does not charge any fee from the applicants and we do not allow other companies also to do so.
About Company
RealPage, Inc. is a leading global provider of software and data analytics to the real estate industry, offering solutions designed to empower property management professionals to improve operational efficiencies and enhance the tenant experience. Founded in 1998 and headquartered in Richardson, Texas, RealPage's suite of services includes property management software, asset optimization, utility management, online leasing, and resident services. With a focus on innovation and customer service, RealPage assists clients in managing and growing their real estate portfolios, emphasizing the importance of data-driven decisions in the dynamic real estate market. Serving various sectors, including residential, commercial, and vacation rentals, RealPage is committed to transforming the way real estate is managed in a digital age.
Important dates & deadlines?
Application Deadline
30 Mar 26, 02:55 PM IST
Similar Jobs
View All
Don't Miss out any Updates
Subscribe now for the latest job alerts
and never miss an update
QA Engineer III-AI
Share with
