Senior Software Engineer, Agent Infrastructure

Stack

Responsibilities

• ▹Build secure execution environments for agent-generated code
• ▹Design identity, authentication and trust boundaries for agents
• ▹Handle model routing and orchestration across different model types and environments
• ▹Implement rate limiting, quotas and resource management for long-running agent workflows
• ▹Build state, memory and filesystem abstractions for agents
• ▹Turn emerging ML research ideas into production-ready infrastructure
• ▹Prototype and evaluate new technologies, deciding what moves into production with research teams

Requirements

• ▹Experience building production ML infrastructure with strong systems fundamentals
• ▹Hands-on work with agentic systems, multi-agent workflows, or agent development frameworks
• ▹Familiarity with model routing and LLM provider frameworks across different model types and environments
• ▹Experience with scalable, fault-tolerant distributed systems and Kubernetes
• ▹A track record of moving quickly on prototypes and making good productionization decisions

Nice to have

• ▹Experience across on-prem, private cloud and public cloud environments
• ▹Familiarity with storage systems, embedded databases or filesystem abstractions
• ▹Experience with code execution sandboxes such as gVisor, Firecracker, Kata or WASM runtimes
• ▹Interest in emerging ML infrastructure, edge inference or browser-native models
• ▹Open-source contributions to LLM or agent infrastructure projects
• ▹Experience with identity, workload auth or capability-based security systems

Soft skills

What we offer

• ▹Weekly lunch stipend of $75 (or local equivalent)
• ▹Full health and dental benefits, including a separate mental health budget
• ▹Retirement plan contributions (RRSP/401K/pension scheme)
• ▹100% parental leave top-up for up to 6 months, for either parent
• ▹Annual enrichment benefits: arts & culture, fitness/wellness, quality time, workspace credit
• ▹Education & learning stipend for conferences, courses and coaching
• ▹6 weeks of paid vacation (30 working days)
• ▹Budget for travel to other offices, plus an annual company offsite
• ▹$500 home office setup stipend

About the company

Cohere is the leading security-first enterprise AI company. It builds cutting-edge foundation AI models and end-to-end products designed to solve real-world business problems, co-headquartered in Toronto and San Francisco with key offices in London, New York City, Montreal, Seoul, Germany and Paris.