Data Annotation Specialist, Software Engineering

Stack

Responsibilities

• ▹Evaluate the model's ability to respond to coding requests, workflows, and code base-related questions using available tools
• ▹Assess agent trajectories and model capabilities for code generation and debugging requests
• ▹Prompt models to complete complex coding tasks and review the accuracy of generated responses
• ▹Label, proofread, and improve machine-written and human-written software engineering-related outputs
• ▹Report quality and performance trends related to model/agent behaviour and project assignments

Requirements

• ▹Subject matter expertise in software engineering — studied in the field and 3-5 years of relevant industry experience
• ▹Proficient knowledge of Python and the following languages: Java, JavaScript, Go, and SQL (any dialect)
• ▹Prior experience designing, building, and deploying APIs
• ▹Prior experience with code agents (OpenCode, Claude Code, Codex, Cursor) or evaluating agent trajectories is a big plus

Nice to have

• ▹Deeper experience with code agents and evaluating agent trajectories

Soft skills

What we offer

• ▹Fully remote, BYOD (bring-your-own-device) work within Canada
• ▹40 CAD/hour contract rate, minimum 16 hours per week
• ▹12-month contract with performance incentives
• ▹Independent contractor status — may work with multiple clients simultaneously

About the company

Cohere is the leading security-first enterprise AI company, building cutting-edge foundation AI models and end-to-end products that solve real-world business problems. A global technology company co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany, and Paris.