Software Engineer - Voice AI (Inference Runtime)

Stack

Responsibilities

• ▹Build a world-class model serving stack for state-of-the-art open-source voice models
• ▹Reduce end-to-end and tail latency (p95/p99), increase throughput and improve GPU efficiency through profiling and runtime tuning
• ▹Build large-scale, real-time infrastructure for multi-model voice agents, orchestrating STT, TTS and agent components with streaming I/O
• ▹Design tight training and inference iteration loops for custom voice model development
• ▹Own Voice AI product areas end-to-end, from architecture through long-term production operations
• ▹Collaborate with Forward Deployed Engineers and Model Performance Engineers
• ▹Mentor teammates through code reviews and design docs

Requirements

• ▹Bachelor's degree or higher in Computer Science or a related field
• ▹Proven track record owning production-grade, real-time, large-scale systems where tail latency (p99) matters
• ▹Proficient coding ability in one or more popular programming languages; Python is a plus
• ▹Good product taste, particularly for developer-oriented tools
• ▹Interest in ML/AI infrastructure and willingness to learn
• ▹Comfortable using AI coding assistants (e.g. Claude Code, Codex, Cursor) as a daily productivity multiplier

Nice to have

• ▹Experience implementing pipeline-level model runtime optimizations such as dynamic batching, async scheduling, or decode-side throughput improvements
• ▹Experience building developer platforms: SDKs, CLIs, APIs and self-serve workflows
• ▹Experience with containerization and orchestration technologies (Docker, Kubernetes), service meshes or distributed scheduling
• ▹Familiarity with speech/audio ML models (STT, TTS, speech-to-speech)
• ▹Familiarity with model-serving runtimes (vLLM, TensorRT, ONNX)
• ▹Systems-level performance profiling across host-device boundaries (e.g. PyTorch Profiler), diagnosing GPU utilization issues
• ▹Exposure to customer-facing engineering: pre-sales prototyping, technical discovery

Soft skills

What we offer

• ▹Competitive compensation, including meaningful equity
• ▹100% coverage of medical, dental and vision insurance for employee and dependents
• ▹Flexible PTO policy including company-wide Winter Break (offices closed from Christmas Eve to New Year's Day)
• ▹Paid parental leave
• ▹Fertility and family-building stipend through Carrot
• ▹Company-facilitated 401(k)
• ▹Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities

About the company

Baseten powers mission-critical inference for the world's most dynamic AI companies, including Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By combining applied AI research, flexible infrastructure and seamless developer tooling, Baseten helps companies at the frontier of AI bring cutting-edge models into production. The company is growing quickly and recently closed a $1.5B Series F round led by Altimeter Capital, Conviction Partners and Spark Capital.