Software Engineer (AI Evaluation, Contract)

Stack

Responsibilities

• ▹Simulate authentic developer activities (version control, review, CI/CD)
• ▹Grade technical correctness of outputs in real development tools
• ▹Design and maintain reproducible test environments (repos, CI flows)
• ▹Configure and document integrations and authentication flows
• ▹Investigate and document undocumented product behaviors
• ▹Participate in calibration sessions for consistent grading

Requirements

• ▹At least 3 years of professional software engineering experience
• ▹Strong Git/GitHub workflow command (PRs, review, branching, diffs)
• ▹CI/CD systems experience, ideally GitHub Actions
• ▹Advanced scripting in Python or Bash
• ▹Understanding of REST APIs, OAuth, and webhook integrations

Nice to have

• ▹Experience leveraging AI assistants or agent tools in dev tasks
• ▹Familiarity with Model Context Protocol (MCP)
• ▹Background in technical QA or data annotation

Soft skills

About the company

micro1 runs evaluation projects that train next-generation AI systems, engaging domain experts to test real-world workflows.