Senior Member of Technical Staff, Web Data

Stack

Responsibilities

• ▹Maintain large-scale pipelines for processing web corpora
• ▹Work on filtering and quality-scoring systems to identify high-value web documents
• ▹Analyze web data composition across domains, languages, and time periods
• ▹Develop and maintain highly-performant deduplication pipelines
• ▹Collaborate with researchers and engineers to ensure data pipelines meet the demands of cutting-edge language models

Requirements

• ▹Strong software engineering skills, proficiency in Python, experience building data pipelines
• ▹Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools
• ▹Experience working with large-scale web datasets
• ▹Knowledge of data quality assessment techniques and experimentation with data mixtures
• ▹A passion for bridging research and engineering to solve complex data-related challenges in AI model training

Nice to have

• ▹Paper at top-tier venues (NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or local currency equivalent)
• ▹Full health and dental benefits, including a separate mental health budget
• ▹RRSP matching, 401K, Pension Scheme
• ▹100% Parental Leave top-up for up to 6 months
• ▹Annual enrichment benefits, education & learning stipend
• ▹6 weeks of paid vacation (30 working days)
• ▹Fully remote-friendly (EST/EU timezone), travel budget to other offices
• ▹$500 home office stipend

About the company

Cohere is the leading security-first enterprise AI company, building cutting-edge foundation AI models and end-to-end products that solve real-world business problems. A global technology company co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany, and Paris.