Site Reliability Engineer, Inference Infrastructure

Stack

Responsibilities

• ▹Build self-service systems that automate managing, deploying and operating services, including custom Kubernetes operators for language model deployments
• ▹Automate environment observability and resilience
• ▹Enable all developers to troubleshoot and resolve problems themselves
• ▹Take the steps required to hit defined SLOs, including participation in an on-call rotation
• ▹Build strong relationships with internal developers and influence the infrastructure team's roadmap
• ▹Develop the team through knowledge sharing and an active review process

Requirements

• ▹5+ years of engineering experience running production infrastructure at scale
• ▹Experience designing large, highly available distributed systems with Kubernetes and GPU workloads
• ▹Experience with Kubernetes development, production coding and support
• ▹Experience with GCP, Azure, AWS, OCI or multi-cloud/on-prem/hybrid serving
• ▹Experience designing, deploying, supporting and troubleshooting complex Linux-based computing environments
• ▹Experience in compute/storage/network resource and cost management
• ▹Excellent collaboration and troubleshooting skills to build mission-critical systems

Nice to have

• ▹Familiarity with the computational characteristics of accelerators (GPUs, TPUs, custom accelerators) and how they affect inference latency and throughput
• ▹Strong understanding or working experience with distributed systems
• ▹Experience in Golang, C++ or other languages designed for high-performance, scalable servers

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or equivalent in local currency)
• ▹Full health and dental benefits, including a separate mental health budget
• ▹RRSP matching, 401K, pension scheme
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits (arts & culture, fitness/wellness, workspace improvement)
• ▹Education and learning stipend for conferences and courses
• ▹6 weeks of paid vacation (30 working days)
• ▹Travel budget to other offices for remote employees, plus an annual company offsite
• ▹$500 home office stipend

About the company

Cohere is a leading security-first enterprise AI company building cutting-edge foundation AI models and end-to-end products that solve real-world business problems for enterprises building AI systems. The team spans researchers, engineers and designers, co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany and Paris.