Lead Member of Technical Staff, Inference Infrastructure

Stack

Responsibilities

• ▹Provide technical leadership across multiple teams, driving the architecture and strategy for deploying optimized NLP models to production in low-latency, high-throughput, high-availability environments
• ▹Serve as a key point of contact for customers, leading the design of customized deployments to meet their specific needs
• ▹Mentor engineers to raise the technical bar across the team
• ▹Develop, deploy and operate the AI platform that delivers Cohere's large language models through easy-to-use API endpoints, as part of the Model Serving team

Requirements

• ▹8+ years of engineering experience running production infrastructure at large scale, with a track record of technical leadership
• ▹Demonstrated experience leading the architecture and design of large, highly available distributed systems running Kubernetes and GPU workloads
• ▹Deep expertise with Kubernetes development and production support, including setting team-wide standards and best practices
• ▹Extensive experience across GCP, Azure, AWS, OCI and multi-cloud on-prem/hybrid serving environments, able to guide strategic infrastructure decisions
• ▹Proven ability to lead the design, deployment, support and troubleshooting of complex Linux-based computing environments at scale
• ▹Experience owning compute/storage/network resource and cost management at an organizational level, including optimization strategies
• ▹Strong expertise in the computational characteristics of accelerators (GPUs, TPUs and/or custom accelerators), leveraging them to improve latency and throughput at scale
• ▹Deep knowledge of distributed systems, with experience establishing patterns and practices across engineering teams
• ▹Proficiency in Golang, C++ or other languages for high-performance, scalable servers, with the ability to set coding standards and conduct senior-level technical reviews

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or local equivalent)
• ▹Full health and dental benefits, including a separate mental health budget
• ▹RRSP matching, 401(k), or pension scheme depending on location
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits (arts & culture, fitness/wellness, quality time, workspace improvement credit)
• ▹Education & learning stipend for conferences, courses and coaching
• ▹6 weeks of paid vacation (30 working days)
• ▹Travel budget to other offices for remote employees, plus an annual company offsite
• ▹$500 home office stipend, or a co-working benefit for those not near an office

About the company

Cohere is a leading security-first enterprise AI company that builds cutting-edge foundation models and end-to-end AI products for real-world business problems. The company is co-headquartered in Toronto and San Francisco, with additional offices in London, New York City, Montreal, Seoul, Germany and Paris. The Model Serving team is responsible for developing, deploying and operating the AI platform that delivers Cohere's large language models through easy-to-use API endpoints.