Product Manager, Inference Platform

Stack

Responsibilities

• ▹Own how workloads scale and where they land — autoscaling to demand and a unified placement policy across region, compliance and capacity preference
• ▹Make production inference reliable by default — every request reaches a healthy replica, rolling deploys never drop traffic, region-aware routing with multi-region/active-active support
• ▹Build the release engine behind safe rollouts — traffic-shifting for canary/shadow/A/B testing, warm-ups, drain and probes
• ▹Push the cost/performance frontier for serving AI at scale — latency, throughput, uptime, cost-efficiency, and reduced MTTR through self-serve incident management
• ▹Drive cross-team roadmaps and their dependencies in a category that doesn't fully exist yet

Requirements

• ▹8+ years in product management, including deep experience with infrastructure, distributed systems or ML serving
• ▹Fluent reasoning about scaling, routing, failover and the cost/performance frontier, earning the respect of staff engineers
• ▹Ownership of capabilities end to end, backend through UX, rather than a single slice
• ▹Ability to drive cross-team roadmaps and their dependencies, defining a category that doesn't fully exist yet

Nice to have

• ▹Hands-on experience with GPU infrastructure, Kubernetes or serving frameworks like vLLM, TensorRT-LLM or SGLang

Soft skills

What we offer

• ▹Competitive compensation, including meaningful equity
• ▹100% coverage of medical, dental and vision insurance for employee and dependents
• ▹Flexible PTO policy including company-wide Winter Break (offices closed from Christmas Eve to New Year's Day)
• ▹Paid parental leave
• ▹Fertility and family-building stipend through Carrot
• ▹Company-facilitated 401(k)
• ▹Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities

About the company

Baseten powers mission-critical inference for the world's most dynamic AI companies, including Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By combining applied AI research, flexible infrastructure and seamless developer tooling, Baseten helps companies at the frontier of AI bring cutting-edge models into production. The company is growing quickly and recently closed a $1.5B Series F round led by Altimeter Capital, Conviction Partners and Spark Capital.