← Back to list
Job · Senior

Senior Research Scientist, Model Evaluation

Cohere
AI / ML Engineer • Senior • Remote • Full-time • 📍 London
📄 View original listing (EN)

Senior Research Scientist, Model Evaluation at Cohere, responsible for creating next-generation evaluation methods and infrastructure to measure LLM progress, building ambitious benchmarks and advancing the state-of-the-art in LLM evaluation.

Responsibilities

  • ▹Create ambitious new evaluation benchmarks that push the limits of what models can accomplish
  • ▹Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations
  • ▹Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges, refining LLM-based data synthesis pipelines and improving evaluation efficiency
  • ▹Build scalable and reusable tools for digging into model performance

Requirements

  • ▹Enjoy rapidly building prototypes that demonstrate the boundaries of LLM capabilities and developing resources to measure them
  • ▹Have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality
  • ▹Obsessive about rigorously measuring AI capabilities and ensuring measurements align with the capabilities you care about
  • ▹Strong software engineering skills

Soft skills

Rigor and attention to detailCross-functional collaborationStrong analytical mindset

What we offer

  • ▹Weekly lunch stipend of $75/£75 (or local equivalent)
  • ▹Full health and dental benefits, including a separate mental health budget
  • ▹RRSP matching, 401K, Pension Scheme
  • ▹100% Parental Leave top-up for up to 6 months for either parent
  • ▹Annual enrichment benefits and an education & learning stipend
  • ▹6 weeks of paid vacation (30 working days)
  • ▹Travel budget to other offices for remote staff plus an annual company offsite
  • ▹$500 home office stipend; co-working benefit for those not near an office
Languages: angol