Senior Technical Program Manager (Engineering) - AI Tooling & Systems

Stack

Responsibilities

• ▹Own end-to-end delivery of AI infrastructure programs (training pipelines, experiment tracking, inference serving, production monitoring)
• ▹Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling
• ▹Serve as connective tissue between ML research, ML engineering, product, and data teams
• ▹Drive cost and latency optimization for real-time inference workloads
• ▹Build lightweight internal tools and processes to accelerate ML iteration cycles
• ▹Identify and resolve technical bottlenecks in training pipelines and serving infrastructure
• ▹Work closely with ML practitioners to translate research breakthroughs into scalable systems

Requirements

• ▹5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling
• ▹Strong technical acumen in ML systems, ideally with hands-on ML/systems engineering experience
• ▹Experience coordinating cross-functional ML programs (training → evaluation → serving → monitoring)
• ▹Proven ability to translate ML/research requirements into robust, scalable infrastructure
• ▹Comfortable working in ambiguity and navigating complex technical tradeoffs (accuracy vs. latency vs. cost)
• ▹Excellent communication with technical and non-technical stakeholders

Nice to have

• ▹Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar)
• ▹Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching)
• ▹Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC)
• ▹Background in feature stores, vector databases, or real-time ML systems
• ▹Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure
• ▹Hands-on with PyTorch, CUDA, Hugging Face, or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML)

Soft skills

About the company

Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time speech-to-text (STT) and text-to-speech (TTS) APIs and production-grade voice agents. More than 200,000 developers and 1,300+ organizations, including Twilio, Cloudflare, and Sierra, build on Deepgram.