← Back to list
Job · Senior

Senior ML Systems Engineer, Frameworks & Tooling

Cohere
AI / ML Engineer • Senior • Remote • Full-time • 📍 London
📄 View original listing (EN)

Senior ML Systems Engineer at Cohere building, maintaining and evolving the training framework that powers frontier-scale language models. The role sits at the intersection of large-scale training, distributed systems and HPC infrastructure, owning core components and tooling that connect research ideas to thousands of GPUs.

Stack

DockerKubernetesPyTorch

Responsibilities

  • ▹Build and own the training framework responsible for large-scale LLM training
  • ▹Design distributed training abstractions (data/tensor/pipeline parallelism, FSDP/ZeRO strategies, memory management, checkpointing)
  • ▹Improve training throughput and stability on multi-node clusters (e.g. GB200/300, AMD, H200/100)
  • ▹Develop and maintain tooling for monitoring, logging, debugging and developer ergonomics
  • ▹Collaborate with infra teams to ensure cluster, container environments and hardware configurations support high-performance training
  • ▹Investigate and resolve performance bottlenecks across the ML systems stack
  • ▹Build robust systems for reproducible, debuggable, large-scale runs

Requirements

  • ▹Strong engineering experience in large-scale distributed training or HPC systems
  • ▹Deep familiarity with JAX internals, distributed training libraries, or custom kernels/fused ops
  • ▹Experience with multi-node cluster orchestration (Slurm, Ray, Kubernetes or similar)
  • ▹Comfort debugging performance issues across CUDA/NCCL, networking, IO and data pipelines
  • ▹Experience with containerized environments (Docker, Singularity/Apptainer)
  • ▹A track record of building tools that increase developer velocity for ML teams
  • ▹Excellent judgment around trade-offs (performance vs complexity, research velocity vs maintainability)
  • ▹Strong collaboration skills with infra, research and deployment teams

Nice to have

  • ▹Experience training LLMs or other large transformer architectures
  • ▹Contributions to ML frameworks (PyTorch, JAX, DeepSpeed, Megatron, xFormers, etc.)
  • ▹Familiarity with evaluation and serving frameworks (vLLM, TensorRT-LLM, custom KV caches)
  • ▹Experience with data pipeline optimization, sharded datasets or caching strategies
  • ▹Background in performance engineering, profiling or low-level systems
  • ▹Paper at top-tier venues (e.g. NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)

Soft skills

Strong collaborationExcellent judgment on trade-offsEnd-to-end ownership

What we offer

  • ▹Weekly lunch stipend of $75/£75 (or local equivalent)
  • ▹Full health and dental benefits, including a separate mental health budget
  • ▹RRSP matching, 401K, Pension Scheme
  • ▹100% Parental Leave top-up for up to 6 months for either parent
  • ▹Annual enrichment benefits and an education & learning stipend
  • ▹6 weeks of paid vacation (30 working days)
  • ▹Travel budget to other offices for remote staff plus an annual company offsite
  • ▹$500 home office stipend; co-working benefit for those not near an office
Languages: angol