An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
Job Description
- Experience: 8+ years in Software QA, with at least 2-3 years focused on testing or evaluating AI/ML systems, conversational agents, or Large Language Models (LLMs).
- Safety Evals Expertise (Mandatory): Direct experience in designing and executing safety evaluations (red teaming, adversarial testing), bias detection, and measuring toxicity/harmful content in generative AI models.
- Agent/LLM Evals: Proven experience developing and running general evaluations (Evals) for LLM-powered applications knowing libraries like PyTest (Must)
- Google ADK Familiarity (Mandatory): Direct or strong conceptual understanding of the Google Agent Development Kit (ADK) and its components.
- Programming: Strong proficiency in Python is mandatory for script development, data processing, and automation.
- Cloud & MLOps: Familiarity with Google Cloud Platform (GCP) services relevant to AI/ML (e.g., Vertex AI) and integrating testing into MLOps workflows.
- Communication: Excellent analytical, problem-solving, and communication skills to articulate complex agent behaviors, risks, and safety findings clearly.
- Tools and Libraries: Langsmith, DeepEval, Ragas, Giskard, Hugging face.
Skills
PythonHugging FaceLLMGenerative AIData EngineeringMLOpsGoogle CloudGCPPyTestManual TestingQACommunicationIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Corporate Finance roles in Remote -India.
One email a day. Unsubscribe any time.

