← Back to list
Job · Principal

Staff AI Engineer - Grafana AI/ML | Canada | Remote

AI / ML Engineer • Principal • Remote • Full-time Canada Canada

Grafana Labs' AI team is hiring a Staff AI Engineer to build LLM- and GenAI-powered features that help users make sense of observability data and detect, triage and resolve incidents.

Responsibilities

  • Develop high-performance AI features to help users detect, triage and resolve incidents using observability data
  • Rapidly prototype, test and validate with real users in an iterative process, including shipping and evolving LLM- or agent-powered workflows
  • Collaborate cross-functionally with data analysts, product managers and designers to shape AI-driven product features
  • Use AI and automation tools to enhance both product functionality and personal development workflow
  • Take full ownership of developed AI solutions, ensuring they are scalable, maintainable and aligned with real user workflows

Requirements

  • Strong engineering skills building production software systems (backend and/or full-stack)
  • Experience with LLMs, prompt engineering, and building applications powered by GenAI
  • Proven track record of delivering software into production that is actively used
  • Exposure to cloud-native environments (e.g. AWS, GCP, Azure)
  • Using observability tools to understand and troubleshoot system behavior

Nice to have

  • Experience building or working with agent frameworks or multi-agent workflows
  • Familiarity with infrastructure/DevOps tooling: Kubernetes, Docker, Terraform or similar
  • Familiarity with model fine-tuning techniques
  • Experience building observability tooling

Soft skills

Self-driven, proactive attitudeExperimental, fast-iteration mindsetCollaborative approach with designers and product managersOpenness to feedbackComfort with ambiguity, ability to define scope

What we offer

  • Restricted Stock Units (RSUs)
  • Company-funded budget for AI tools (e.g. GPT-Codex, Claude Opus, Gemini Pro)
  • 100% remote work in a global team spanning 40+ countries
  • 30 days of annual leave, including 3 shared Grafana Shutdown Days

About the company

Grafana Labs is the company behind the open observability cloud, building Grafana Cloud. It serves more than 35 million users and 7,000+ customers – including Anthropic, Bloomberg, NVIDIA, Microsoft and Salesforce – with 1,600+ team members across 40+ countries, fully remote.

Similar jobs