Senior Engineering Manager, Model Infrastructure

Stack

Responsibilities

• ▹Lead and grow a high-performing engineering team responsible for Harvey's Model Infrastructure platform
• ▹Define the technical roadmap for model reliability, scalability, and operational excellence
• ▹Build highly reliable systems for model provisioning, capacity management, failover, and incident response across multiple AI providers
• ▹Own the multi-provider model platform, including provider integrations, SDK upgrades, API migrations, and onboarding new providers
• ▹Drive the evolution of the Unified Model Controller (UMC) and Model Selector platform for health-, latency-, quality-, compliance-, and cost-based routing
• ▹Improve observability through health dashboards, alerting, token usage analytics, and cost reporting
• ▹Partner with Product Engineering on new model launches, capacity planning, and proactive production monitoring
• ▹Lead initiatives to improve inference efficiency, reduce infrastructure costs, and increase model utilization
• ▹Build the infrastructure foundation for future model training efforts, including data pipelines and model operations
• ▹Partner with executive leadership on long-term AI infrastructure strategy and vendor relationships
• ▹Recruit, mentor, and develop engineering talent

Requirements

• ▹8+ years of software engineering experience, including multiple years managing high-performing engineering teams
• ▹Experience leading teams responsible for large-scale distributed systems or cloud infrastructure
• ▹Strong technical background enabling you to guide architectural decisions and mentor senior engineers
• ▹Experience operating highly available production services with strong reliability focus
• ▹Experience building platforms requiring scalability, observability, automation, and cost optimization
• ▹Strong cross-functional leadership skills across Engineering, Research, Product, and external vendors
• ▹Excellent communication skills and ability to influence technical strategy across organizations
• ▹A passion for building teams and developing engineering talent

Nice to have

• ▹Experience with AI infrastructure, LLM serving, or machine learning platforms
• ▹Experience working with multiple model providers such as OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, or open-source ecosystems
• ▹Experience building inference platforms, model gateways, traffic routing systems, or policy-based serving infrastructure
• ▹Experience with Kubernetes, cloud infrastructure, distributed systems, and large-scale observability platforms
• ▹Experience supporting GPU infrastructure, model training platforms, or ML infrastructure
• ▹Familiarity with data platforms such as Spark, Kafka, Flink, Airflow, or Iceberg
• ▹Experience leading organizations through rapid growth and technical transformation

Soft skills

About the company

Harvey is transforming how legal and professional services operate, combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise, with 1500+ customers in 60+ countries.