Member of Technical Staff (AI Inference Engineer)

Stack

Responsibilities

• ▹Support new models in the inference infrastructure: weight loading, request scheduling, KV-cache management, and API Gateway support
• ▹Port in-house CUDA kernels to NVIDIA's CuTe DSL so they run on GB200 today and are portable to Vera Rubin racks tomorrow
• ▹Develop the internal Rust-based inference server to solve Python pains and keep up with rapidly growing traffic
• ▹Profile and fix performance bottlenecks from network ingress through continuous batching and GPU kernel interleaving
• ▹Build dashboards, alerts, and automated remediation for reliability and observability
• ▹Respond to and learn from production incidents

Requirements

• ▹Deep experience with GPU programming and performance work (CUDA, Triton, CUTLASS, or similar)
• ▹Understanding of modern LLM architectures and how to bring them up reliably in production
• ▹Experience building and operating production distributed systems under real load, ideally performance-critical ones
• ▹Comfortable working across languages and layers: Rust for the serving runtime, Python for model code, CUDA/CuteDSL for kernels
• ▹Ownership of problems end-to-end
• ▹Self-directed, thriving in fast-moving environments where the path isn't laid out
• ▹3+ years of professional software engineering experience with meaningful work on ML inference or high-performance systems
• ▹Familiarity with at least one deep learning framework (PyTorch, JAX, TensorFlow)
• ▹Understanding of GPU architectures (memory hierarchy, warp scheduling, tensor cores)
• ▹Understanding of common LLM architectures and inference optimization techniques (e.g. quantization, speculative decoding, prefill-decode disaggregation)

Nice to have

• ▹Experience with ML compilers and framework internals (PyTorch internals, torch.compile, custom operators)
• ▹Distributed GPU communication: NCCL, NVLink, InfiniBand, RDMA libraries, model/tensor parallelism
• ▹Low-precision inference: INT8/FP8/FP4 quantization, mixed-precision serving
• ▹Profiling and debugging tools: Nsight Compute/Systems, CUDA-GDB, PTX/SASS analysis
• ▹Container orchestration: Kubernetes, GPU scheduling, autoscaling inference workloads

Soft skills

About the company

Perplexity is building the future of AI-powered search and agent experiences through its Sonar models, Deep Research Agent, Comet Agent, and Search products.