Software Engineer, GPU Infrastructure (HPC)

Stack

Responsibilities

• ▹Deploy and manage Kubernetes-based GPU/TPU superclusters across multiple clouds, ensuring high throughput and low-latency performance for AI workloads
• ▹Collaborate with cloud providers to fine-tune infrastructure for cost efficiency, reliability and performance, leveraging technologies like RDMA, NCCL and high-speed interconnects
• ▹Proactively identify and resolve infrastructure bottlenecks, performance degradation and system failures to ensure minimal disruption to AI/ML workflows
• ▹Design intuitive self-service tools and workflows that allow researchers to monitor, debug and optimize their training jobs independently
• ▹Work closely with AI researchers to understand emerging needs (e.g. JAX, PyTorch, distributed training) and translate them into robust, scalable infrastructure solutions
• ▹Advocate for observability, automation and infrastructure-as-code (IaC) across the organization
• ▹Share expertise through code reviews, documentation and cross-team collaboration

Requirements

• ▹Deep expertise in ML/HPC infrastructure: GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow) and high-performance computing environments
• ▹Proven ability to deploy, manage and troubleshoot cloud-native Kubernetes clusters for AI workloads at scale
• ▹Strong programming skills in Python (for ML tooling) and Go (for systems engineering)
• ▹Familiarity with Linux internals, RDMA networking and performance optimization for ML workloads
• ▹A track record of working closely with AI researchers or ML engineers to solve infrastructure challenges
• ▹Self-directed problem-solving in fast-paced environments

Soft skills

What we offer

• ▹Weekly lunch stipend ($75/£75 or equivalent in local currency)
• ▹Full health and dental benefits, including a separate mental health budget
• ▹RRSP matching, 401K, pension scheme
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits (arts & culture, fitness/wellness, workspace improvement)
• ▹Education and learning stipend for conferences and courses
• ▹6 weeks of paid vacation (30 working days)
• ▹Travel budget to other offices for remote employees, plus an annual company offsite
• ▹$500 home office stipend

About the company

Cohere is a leading security-first enterprise AI company building cutting-edge foundation AI models and end-to-end products that solve real-world business problems for enterprises building AI systems. The team spans researchers, engineers and designers, co-headquartered in Toronto and San Francisco, with offices in London, New York City, Montreal, Seoul, Germany and Paris.