Head of AI Engineering

Stack

Responsibilities

• ▹Agent SDK: turn the existing customer support agent harness into a strongly opinionated internal platform powering AIOS agents; own its architecture, reusable primitives, extension points, developer experience and integration with existing infrastructure
• ▹Jesse & Aegis: become the senior technical owner of Jesse (customer support tool) and work with engineers building Aegis (clinical decision support); extract shared foundations across context, retrieval, memory, orchestration, tools, state and escalation
• ▹Evals & Experimentation: build trustworthy benchmarks using deterministic checks, simulations, model-based graders, human judgment and production outcomes; establish the path from offline evaluation to controlled production experiments
• ▹Production Learning Loop: turn traces, poor resolutions, escalations, incidents, tool failures and successful outcomes into better evals, stronger architecture, improved models and permanent platform capabilities
• ▹Safety & Compliance: make consequential agent actions safe through authorization, validation, idempotency, auditability, recovery and human handoff; encode compliance, privacy, security and regional requirements into the platform
• ▹Models & Economics: own model selection, routing, fallbacks, caching and the ~$200k monthly model spend; when evidence supports it, lead data preparation, fine-tuning, evaluation and AIOS-controlled deployment of specialized open-weight models
• ▹Reliability: own the shared runtime in production, including tracing, observability, testing, provider resilience, capacity and incident response; be the senior engineering owner when an AI system behaves unsafely, quality regresses or the platform fails
• ▹Technical Leadership: set AIOS's AI architecture and strategy with the VP of Engineering, make the final call on major technical decisions, guide engineers across product pods and stay hands-on writing production code
• ▹Build the Team: inherit one engineer and build the Applied AI team to about five exceptional people during the first year; own technical relationships with leading model providers

Requirements

• ▹8+ years of software engineering experience and an active production contributor
• ▹At least a bachelor's degree in Computer Science, Machine Learning or a closely related technical field
• ▹Personally built and shipped an exceptional agentic system used by real customers: it reasoned across multiple steps, used tools, changed state and operated under real production constraints
• ▹Agent architecture: reason deeply about harnesses, orchestration, context construction, retrieval, memory, state, tool design, structured workflows and error recovery
• ▹Evals: built or meaningfully owned evaluation systems for probabilistic products; understand dataset construction, evaluator design, simulations, regression detection, noisy metrics and the relationship between offline performance and production outcomes
• ▹Software engineering: strong systems fundamentals across APIs, distributed systems, concurrency, queues, databases, observability, failure modes and production reliability
• ▹Consequential actions: strong judgment around authorization, validation, idempotency, state transitions, auditability, recovery and escalation
• ▹Model judgement: understand the capabilities and limitations of current frontier and open-weight models; know when the problem is the model and when it is context, tools, data, orchestration or evaluation
• ▹Open-weight models: enough technical depth to lead fine-tuning and AIOS-controlled deployment when evidence supports it (prior production deployment is not required)
• ▹Leadership: successfully led and managed a small technical engineering team, set clear direction, raised the quality bar, developed strong engineers and addressed underperformance
• ▹Technical authority: strong engineers trust your judgment; you make difficult decisions, explain trade-offs clearly and push back on unsound approaches
• ▹Communication: explain difficult technical ideas to engineers, product leaders, clinicians and executives without flattening important details
• ▹Independence: create clarity in ambiguous environments and make high-quality decisions without hand-holding
• ▹Builder: still write production code and lead from inside the work
• ▹Ownership: when quality drops, costs spike, tools fail or providers degrade, you take responsibility for reaching the outcome rather than assigning blame

Nice to have

• ▹Agent platforms: built runtimes, SDKs, harnesses, tool layers, evaluation platforms or shared AI infrastructure used by other engineers
• ▹Customer agents: built high-volume customer-service, commerce or transactional agents across complex, multi-step customer journeys
• ▹High-stakes systems: worked on healthcare, financial, insurance or other systems where correctness, traceability and careful rollout matter
• ▹Model adaptation: fine-tuned, distilled, evaluated or deployed an open-weight model for a specific production workflow
• ▹Long-term memory: built durable memory, context compression, personalization or agents operating across sessions
• ▹Real-time systems: worked on voice agents, streaming systems or other latency-sensitive AI experiences
• ▹Provider relationships: worked directly with frontier model providers on evaluations, technical issues, capacity, pricing or early access
• ▹Talent: a strong nose for exceptional AI engineers and how to create an environment where they do their best work
• ▹Research fluency: translate relevant research into reliable production systems without confusing novelty with progress
• ▹Figure it out: move from debugging a production trace to redesigning an eval, reviewing an agent abstraction or handling a provider incident

Soft skills

What we offer

• ▹Compensation: $200k-$400k/yr plus equity
• ▹Full-time, fully remote role; async work in the timezone of your choice, regularly available until midday Pacific Time
• ▹Comprehensive medical insurance (if appropriate)
• ▹Paid time off with a yearly minimum (at least 2 weeks per year + local requirements)

About the company

AIOS is building the first full-stack AI doctor. It serves over 150k patients per month in the UK through its main brand Bolt Pharmacy, and is a profitable, founder-led company.