Member of Technical Staff, Data Analysis and Evaluation

Stack

Responsibilities

• ▹Design and oversee data collection tasks, including supporting human annotators and ensuring data quality
• ▹Develop and apply statistical methods to evaluate dataset quality and reliability
• ▹Analyse and assess the generalisability and robustness of ML systems across diverse use cases
• ▹Train and fine-tune LLMs on distributed training infrastructures
• ▹Conduct experiments to evaluate model performance and identify areas for improvement

Requirements

• ▹Extremely strong software engineering skills
• ▹Strong expertise designing and conducting data collection tasks, including working with human annotators
• ▹Strong statistical skills and experience evaluating experiments related to data collection and model performance
• ▹Experience analysing datasets for quality, bias and suitability for training ML models
• ▹Hands-on experience training LLMs on distributed training infrastructures
• ▹Proficiency in Python and ML frameworks such as PyTorch, TensorFlow, JAX

Nice to have

• ▹One or more papers at top-tier venues such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or local equivalent)
• ▹Full health and dental benefits, including a mental health budget
• ▹RRSP matching, 401(k) or pension scheme
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits (arts & culture, fitness/wellness, workspace improvement credit)
• ▹Education & learning stipend for conferences, courses and coaching
• ▹6 weeks of paid vacation (30 working days)
• ▹Travel budget to other offices plus an annual company offsite
• ▹$500 home office stipend and co-working benefit for remote employees

About the company

Cohere is a leading security-first enterprise AI company, building frontier foundation models and end-to-end AI products for real-world business problems. It is a global team co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany and Paris.