← Back to list
Job · Senior

Senior Software Engineer - LLM Ops & Evals

Software Engineer • Senior • Hybrid • Full-time • Netherlands Amsterdam, Netherlands

Every AI call in DataSnipper goes through your team's gateway: you own how inference is routed, how it fails over and what it costs, plus the platform used to measure AI quality.

Responsibilities

  • ▹Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers
  • ▹Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code
  • ▹Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run
  • ▹Own the infrastructure for async and long-running AI workloads
  • ▹Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case
  • ▹Take part in the on-call rotation, run incidents, and close the follow-ups
  • ▹Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO
  • ▹Define and maintain clean integration contracts between the platform and the teams that consume it
  • ▹Partner with product and ML engineers to turn their requirements into self-service platform capabilities

Requirements

  • ▹5+ years in backend or platform engineering, with strong production Python
  • ▹Experience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution
  • ▹Experience with cloud at the infrastructure level and infrastructure as code (Azure and GCP with Terraform)
  • ▹Experience running a shared service in production: on-call, incidents, postmortems, SLOs
  • ▹Hands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts
  • ▹Comfort with privacy and compliance work: PII handling, anonymisation, retention, access control
  • ▹Experience building platform or shared-service capabilities consumed by multiple internal teams

Nice to have

  • ▹Experience with LLM or agent evaluation
  • ▹Temporal or another durable workflow engine
  • ▹Self-hosted inference, capacity planning, load testing
  • ▹Synthetic data or document anonymisation pipelines
  • ▹Document AI: VLMs, OCR, structured extraction and the metrics that go with it
  • ▹Domain experience in audit, accounting or fintech

Soft skills

Ownership: you own work end-to-end without close supervisionGrowth mindset and balanced feedbackBuilding cross-functional relationshipsAdaptability in ambiguitySound judgment, balancing speed and accuracy

About the company

DataSnipper is a bootstrapped $1B company automating audit and finance workflows, with 600,000+ users across 180+ countries. Its Excel Agent brings AI directly into Excel with full traceability.

Similar jobs