Trade Compliance Expert For AI Benchmarking

A technology company is building benchmarks that test whether AI agents can handle real professional work from start to finish. They need hands-on compliance practitioners to supply what only experience can: realistic processes, messy real-world data, and the judgment to know when an answer is actually right.

You will turn tax and trade compliance processes you have run yourself into expert-level, terminal-based tasks. Each task includes its input files, a working reference solution, and an automated check that proves whether the agent applied the rules correctly. If you know where analysts typically go wrong in a VAT reconciliation, a tariff classification, or an Intrastat filing, this project puts that knowledge to work.

This is a remote, task-based contract. You are paid per approved task (see How to Apply for compensation details), and you choose how many tasks you take on.

What Youll Do

• Turn real compliance workflows into benchmark tasks. Start from a process you have executed, such as indirect tax determination, customs valuation, tariff classification, or Intrastat reporting, and turn it into a task an AI agent must solve end to end.
• Build the task environment. Define the files, data, and systems each task runs on, such as ERP exports, CSV or JSON files, and databases, so it mirrors real analyst work.
• Write a working reference solution in Python (or another language) and the command line that takes raw inputs all the way to a correct final output.
• Create automated verification that proves the result is objectively correct, and document how a mistake against the written rules would be caught.
• Capture the hard parts: edge cases, data inconsistencies, and the common analyst errors that make the process difficult in practice.
• Refine tasks based on reviewer feedback, aiming for solvable, expert-level challenges that take roughly 60–75 minutes to complete end to end.

What You Bring

• 3+ years doing compliance work hands-on (not only studying, selling, managing, auditing, or teaching it), or a PhD plus active research in the subfield.
• Hands-on work within the last 3 years, and the ability to name the systems, tools, data, and rules you used.
• Personal experience executing a rules-based process where each case is decided by written regulations, tariffs, or SOPs.
• Experience working inside ERP systems of record (SAP or similar) and with their exports and reports.
• The ability to explain a processs edge cases, data inconsistencies, and common errors, and how a mistake would be caught.
• Comfort with Python (or another language), the command line, and CSV/JSON files, APIs, or databases.

Nice to Have

• Experience automating or auditing a compliance process with scripts, SQL, RPA, or rules engines
• Experience in regulated environments such as EU trade, including Intrastat filings
• 5+ years in the same process family
• Experience with Docker, Git, and writing automated tests
• Advanced Excel and experience with regulatory calculators