AI Tester & Evaluator (LLM Stumping)

About the Role

We are looking for experienced AI Testers and Evaluators who are good at two things:

1. Creating smart prompts to stump (challenge and trick) AI models like LLMs.
2. Carefully reviewing and labeling the AIs answers.

This is not just regular data annotation work. You will spend most of your time thinking creatively to find ways to make the AI struggle or make mistakes, then clearly rate and explain its responses. Your work will help make AI models better, safer, and more reliable.

Key Responsibilities

• Write clever and tricky prompts designed to stump the AI and discover its weak points.
• Review the AIs answers and annotate (label) them for quality, accuracy, safety, and usefulness.
• Create test cases that go beyond normal questions to find hidden problems in the AI.
• Write clear notes and reports explaining why the AI succeeded or failed.
• Work with the AI development team to share your findings and improve the models.
• Help build better training data by combining creative testing with careful evaluation.

Requirements

• 3+ years of experience in AI evaluation, testing, or data annotation.
• Strong experience in creating prompts that stump LLMs (this is the most important skill).
• Good attention to detail and consistency in reviewing AI responses.
• Excellent communication skills — you must be able to explain your thoughts clearly in writing.
• Good understanding of how AI models work and where they usually go wrong.
• Ability to be both creative (for stumping) and organized (for annotation work).
• Basic knowledge of Python or annotation tools is a plus.

Important Compliance Rules:

• Any use of AI models (ChatGPT, Claude, Grok, etc.) to generate, train, or assist in creating data for this specific LLM will result in immediate disqualification.
• Any attempt to break compliance, use unethical methods, or violate safety guidelines will eliminate your application.

What We Offer

• Good salary with performance bonuses.
• Chance to work on advanced AI models and see your impact directly.
• Fully remote work with a supportive team.
• Opportunities to learn and grow in AI testing.

Please complete the attached google form - https://forms.gle/a9zdbhbnQMo5ZDh37