ML/Agent Ops Engineer

https://www.tkaccelis.com/

• Design and implement CI/CD pipelines for models, prompts, agents, and supporting infrastructure across development, test, and production environments.
• Build and maintain deployment automation, versioning, rollback mechanisms, environment promotion workflows, and runtime safeguards for AI workloads.
• Set up and operate observability for AI applications and agents, including tracing, monitoring, alerting, token consumption analysis, latency tracking, and incident diagnostics.
• Implement evaluation pipelines and acceptance gates for quality, groundedness, task adherence, safety, and agent-specific behavior.
• Drive prompt lifecycle management, RAG optimization, semantic retrieval tuning, and integration of vector-based or search-based knowledge components where needed.
• Collaborate with security, engineering, and data teams to embed identity, secrets management, compliance controls, and cost optimization into the operating model.
• Operational mindset with strong attention to reliability, security, incident response, and cost-performance trade-offs.

• 5 years to 7 years of relevant experience in DevOps, platform engineering, MLOps, or a closely related role.
• Experience operating production cloud workloads with CI/CD, monitoring, and infrastructure automation.
• Experience with production AI, ML, or agentic workloads is strongly preferred.
• Experience working with high-availability, regulated, or enterprise-scale environments is an advantage.

• Strong experience with Azure, GitHub or Azure DevOps, Docker, Kubernetes, Terraform or Bicep, and infrastructure-as-code patterns.
• Hands-on experience with MLOps, LLMOps, or AgentOps practices for deployment, monitoring, retraining or reevaluation, and controlled release management.
• Strong understanding of observability concepts, including logs, metrics, traces, runtime telemetry, and production diagnostics for AI systems.
• Practical Python skills for automation, tooling, evaluation orchestration, and operational support.
• Familiarity with retrieval-augmented systems, prompt engineering, tool-calling flows, and agent behavior debugging.