Senior Research Engineer, Model Evaluation

Responsibilities

• ▹Develop evaluation benchmarks, datasets and environments to measure the cutting edge of model capabilities
• ▹Conduct research to advance the state of the art in LLM evaluation, including training LLM judges and improving evaluation efficiency
• ▹Build scalable tools for investigating and understanding evaluation results, used across the company including leadership
• ▹Learn from and work with leading researchers and engineers in the field

Requirements

• ▹Track record of building high-quality evaluation resources for LLMs (datasets, simulators, environments)
• ▹Track record of developing new evaluation methods or data, e.g. publications at top-tier conferences or popular benchmarks
• ▹Deep experience building with and around LLMs, including tools for analysing and understanding their performance
• ▹Strong software engineering skills

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or local equivalent)
• ▹Full health and dental benefits, including a mental health budget
• ▹RRSP matching, 401(k) or pension scheme
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits (arts & culture, fitness/wellness, workspace improvement credit)
• ▹Education & learning stipend for conferences, courses and coaching
• ▹6 weeks of paid vacation (30 working days)
• ▹Travel budget to other offices plus an annual company offsite
• ▹$500 home office stipend and co-working benefit for remote employees

About the company

Cohere is a leading security-first enterprise AI company, building frontier foundation models and end-to-end AI products for real-world business problems. It is a global team co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany and Paris.