Staff Software Engineer, GPU Inference

Stack

Responsibilities

• ▹Productionize the GPU inference stack: design, build, and maintain the full GPU prefill path (API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, rack-scale infrastructure)
• ▹Own GPU operational readiness: establish deployment, upgrade, rollback, health-checking, and failure-recovery practices for the AMD GPU fleet
• ▹Drive reliability in production: define SLIs/SLOs, improve fault isolation, graceful degradation, and automated recovery
• ▹Improve inference performance: profile and optimize time to first token, throughput, tail latency, GPU utilization, and memory efficiency
• ▹Optimize model-serving behavior: scheduling, continuous batching, prefix caching, KV-cache management, tensor and expert parallelism
• ▹Debug across system layers: application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware
• ▹Ensure numerical correctness: build validation and regression infrastructure for model quality, precision, and quantization
• ▹Build performance and correctness infrastructure: benchmarks, workload replay tools, profiling automation, dashboards

Requirements

• ▹8+ years of software engineering experience with substantial IC ownership of complex production systems
• ▹Experience building, operating, or optimizing production inference systems for LLMs or similarly demanding GPU workloads
• ▹Strong C++ and Python skills, including multithreading, concurrency, and memory management
• ▹Hands-on experience with a high-performance model-serving framework (e.g. vLLM, SGLang, TensorRT-LLM, Triton Inference Server)
• ▹Strong understanding of GPU execution and performance: async execution, memory movement, synchronization, profiling
• ▹Experience debugging distributed systems across multiple layers
• ▹Experience with Linux, containers, Kubernetes or comparable orchestration, observability, and CI/CD
• ▹Ability to design rigorous benchmarks and interpret noisy performance results
• ▹Strong communication and technical leadership skills
• ▹Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience

Nice to have

• ▹Experience with AMD Instinct accelerators and the ROCm ecosystem (HIP, RCCL, rocprofiler, AMD SMI, etc.)
• ▹Deep CUDA experience that transfers GPU systems knowledge across accelerator platforms
• ▹Experience contributing to vLLM, SGLang, PyTorch, Triton, or TensorRT-LLM
• ▹Experience optimizing prefill-heavy or disaggregated prefill/decode inference architectures
• ▹Understanding of KV-cache transfer, prefix caching, continuous batching, and request scheduling
• ▹Experience with multi-GPU and multi-node inference (tensor, pipeline, expert parallelism, RDMA)
• ▹Experience optimizing Mixture-of-Experts or multimodal models
• ▹Knowledge of GPU kernel optimization, operator fusion, attention kernels, GEMM tuning
• ▹Experience with reduced-precision inference and quantization formats (BF16, FP8, FP4, INT8, INT4)
• ▹Experience building numerical-comparison, determinism, or performance-regression test systems
• ▹Experience collaborating with accelerator vendors, framework maintainers, or open-source communities

Soft skills

About the company

Cerebras Systems builds the world's largest AI chip, 56 times larger than a GPU. The architecture delivers industry-leading training and inference speeds, more than 10 times faster than GPU-based hyperscale cloud inference services. OpenAI recently announced a multi-year partnership with Cerebras to deploy 750 megawatts of capacity.