Freelance Agent Evaluation Engineer

Stack

Responsibilities

• ▹Build realistic developer environments (codebase, infrastructure, tickets, docs) that form a believable development history
• ▹Design tasks from intermediate states of these environments: write the prompt, define what counts as solved, and ensure the task is solvable by an AI agent
• ▹Write tests that accept all valid solution approaches and reject incorrect ones
• ▹Iterate on tasks and tests based on QA feedback

Requirements

• ▹5+ years of software development experience
• ▹Familiarity with Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
• ▹Experience writing tests (functional, integration)
• ▹English proficiency B2+

Soft skills

What we offer

• ▹Flexible, self-paced work
• ▹Up to $40/hr equivalent, depending on level and pace

About the company

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.