Software Engineer - Model Products

Stack

Responsibilities

• ▹Design, build and operate the Model APIs surface with advanced inference capabilities: structured outputs, tool/function calling and multi-modal serving
• ▹Profile and optimize TensorRT-LLM kernels, analyze CUDA kernel performance, implement custom CUDA operators
• ▹Productionize performance improvements across runtimes: speculative decoding, quantization, batching and KV-cache reuse
• ▹Build comprehensive benchmarking frameworks measuring real-world performance across model architectures and hardware configurations
• ▹Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks
• ▹Implement platform fundamentals: API versioning, validation, usage metering, quotas and authentication

Requirements

• ▹3+ years of experience building and operating distributed systems or large-scale APIs
• ▹Proven track record owning low-latency, reliable backend services (rate-limiting, auth, quotas, metering, migrations)
• ▹Infra instincts with performance sensibilities: profiling, tracing, capacity planning, SLO management
• ▹Comfortable debugging complex systems, from runtime internals to GPU execution traces
• ▹Strong written communication, able to produce clear design docs

Nice to have

• ▹Experience with LLM runtimes (vLLM, SGLang, TensorRT-LLM) or contributions to open-source inference engines
• ▹Knowledge of Kubernetes, service meshes, API gateways or distributed scheduling
• ▹Background in developer-facing infrastructure or open-source APIs

Soft skills

What we offer

• ▹Competitive compensation, including meaningful equity
• ▹100% coverage of medical, dental and vision insurance for employee and dependents
• ▹Flexible PTO policy including company-wide Winter Break (offices closed from Christmas Eve to New Year's Day)
• ▹Paid parental leave
• ▹Fertility and family-building stipend through Carrot
• ▹Company-facilitated 401(k)
• ▹Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities

About the company

Baseten powers mission-critical inference for the world's most dynamic AI companies, including Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By combining applied AI research, flexible infrastructure and seamless developer tooling, Baseten helps companies at the frontier of AI bring cutting-edge models into production. The company is growing quickly and recently closed a $1.5B Series F round led by Altimeter Capital, Conviction Partners and Spark Capital.