Member of Technical Staff - Inference Research

Responsibilities

• ▹Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling
• ▹Train custom speculators against real production traffic
• ▹Work directly with customers alongside Forward Deployed Engineers to deploy and tune models
• ▹Carry and expand collaborations with outside research labs (e.g. ZLab, SGLang)
• ▹Work with engineering to turn frontier serving techniques into products
• ▹Help shape the research agenda

Requirements

• ▹A research-leaning or systems background in LLM inference, with demonstrable work
• ▹Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling
• ▹A record of shipping research or systems that other people build on
• ▹Drive to independently take a research bet from idea to result
• ▹Ability to work in-person in the NYC or San Francisco office

Soft skills

About the company

Modal is building a new AI infrastructure layer covering the whole life of an LLM. Customers include category-defining companies like Lovable, Ramp, Cognition, DoorDash, and Suno. They recently raised a $355M Series C at a $4.65B valuation.