Engineering Manager - Model Performance

Stack

Responsibilities

• ▹Lead, mentor, and manage a team of engineers focused on developing and optimizing ML model inference and performance
• ▹Oversee technical strategy and architecture decisions, driving improvements across the engineering organization
• ▹Collaborate with cross-functional teams to ensure seamless integration and scalability of ML models in production
• ▹Dive into the codebase of frameworks like TensorRT, PyTorch, CUDA, and others to identify and solve complex performance bottlenecks
• ▹Drive the development and deployment of large-scale optimization techniques for ML models, especially LLMs
• ▹Own the full lifecycle of projects from inception through delivery, including planning, execution, and resource management
• ▹Foster a collaborative, inclusive team environment that encourages continuous learning and growth

Requirements

• ▹Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field
• ▹5+ years of professional software engineering experience, including at least 2 years in a technical leadership role
• ▹Proven experience managing and mentoring engineering teams
• ▹Expertise in one or more programming languages such as Python, C++, or Go
• ▹In-depth understanding of ML model performance optimization, especially using PyTorch, TensorRT, and CUDA
• ▹Strong knowledge of containerization (Docker) and orchestration systems (Kubernetes)
• ▹Experience with production-level AI/ML solutions, including scaling and deploying large models
• ▹Ability to balance hands-on technical work with team leadership and project management

Nice to have

• ▹Experience enhancing the performance of large language models (LLMs) or similar AI systems
• ▹Familiarity with LLM optimization techniques such as quantization, speculative decoding, or continuous batching
• ▹Deep knowledge of GPU architecture and performance tuning
• ▹Previous experience in a high-growth startup environment

Soft skills

What we offer

• ▹Competitive compensation, including meaningful equity
• ▹100% coverage of medical, dental, and vision insurance for employee and dependents
• ▹Flexible PTO policy, including a company-wide Winter Break (offices closed from Christmas Eve to New Year's Day)
• ▹Paid parental leave
• ▹Fertility and family-building stipend through Carrot
• ▹Company-facilitated 401(k)
• ▹Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities

About the company

Baseten powers mission-critical inference for some of the world's most dynamic AI companies, including Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, Baseten helps companies operating at the frontier of AI bring cutting-edge models into production. The company is growing quickly and recently closed a $1.5B Series F round led by Altimeter Capital, Conviction Partners, and Spark Capital.