← Back to list

Job
· Principal
Staff Software Engineer - Databases, Tempo | Germany | Remote
Software Engineer
• Principal
• Remote
• Full-time
•
Germany
A Staff engineer role on Grafana Labs' Tempo distributed tracing backend, leading the architecture, performance, and AI-product integration of the open-source query engine.
Responsibilities
- ▹Lead multi-quarter technical initiatives from problem framing through rollout (trace aggregation APIs, autoscaling cells, query engine improvements)
- ▹Own the architecture of core Tempo components: ingestion, storage, query, and metrics generation
- ▹Drive design reviews, making trade-offs on performance, cost, and complexity
- ▹Design APIs for both humans and AI agents (structured, deterministic, discoverable)
- ▹Drive operational excellence against concrete SLOs (P99 write latency, incident recurrence, TCO)
- ▹Partner with PMs and sibling teams (App Observability, Asserts, Drilldown, Grafana Assistant)
- ▹Mentor engineers through code review, design feedback, and pairing
- ▹Participate in on-call and contribute to the open-source Tempo project
Requirements
- ▹Track record of leading complex, multi-quarter technical initiatives
- ▹Deep hands-on experience building and operating distributed data systems (ingestion pipelines, storage engines, query execution)
- ▹Strong software craftsmanship, writing clean, robust, performant, maintainable code
- ▹Strong Go skills (or a path to it); deep experience in other systems languages (Rust, C, C++) translates well
- ▹Operational mindset: owning production services, on-call, treating SLOs as a product feature
- ▹Customer focus and pragmatism, short feedback loops
- ▹Leadership through writing and collaboration, clear communication in a remote, async environment
Nice to have
- ▹Experience with tracing, OpenTelemetry, or large-scale observability systems
- ▹Experience designing query languages (SQL/TraceQL-like engines) or programmatically consumable APIs
- ▹Experience with columnar storage formats (e.g. Parquet)
- ▹Experience operating multi-tenant, multi-cell SaaS infrastructure on Kubernetes
- ▹Experience building for AI/LLM consumers: structured APIs, deterministic outputs
- ▹Open-source contribution or maintainership
- ▹Experience as an on-call user of Grafana, Prometheus, Loki, or Tempo
Soft skills
Leadership through design docs and shipped code, not hierarchyClear written communication in an async environmentMentoring and raising the team's technical bar
What we offer
- ▹100% remote work in a global team of 40+ countries
- ▹Company-funded AI tooling budget (frontier models e.g. GPT-5.6, Claude Fable, Gemini Pro 3.1)
- ▹Base compensation in Germany: EUR 109,709 - EUR 131,651
About the company
Grafana Labs is the company behind the open observability cloud, founded on open source and open standards principles. Grafana Cloud is used by 35M+ users and 7,000+ customers including Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce. A 100% remote company with 1,600+ team members across 40+ countries.
Languages: Angol: Felsőfok, Munkavégzés helye: Németország
Similar jobs

Job
· Principal
Staff Software Developer (IAM)
JetBrains
Opentelemetry
💰 Salary: not specified
🏢 On-site
Berlin
🗣️ EN

Job
· Principal
Staff Software Engineer (Agentic Engineering)
Coder
Github Actions
+7
£113,000–£170,000/yr
gross
🌍 Remote
🗣️ EN

Job
· Principal
Staff Software Engineer, Data Platform
Agility Robotics
FlinkCppDatabricks
+11
$191,000–$299,000/yr
gross
🌍 Remote
🗣️ EN

Job
· Principal
Staff Software Engineer, Storage
CoreWeave
CppClickhouse
+3
$207,000–$303,000/yr
gross
🏢 On-site
California
🗣️ EN

Job
· Principal
Staff Software Engineer, Model Infrastructure
Harvey
AI/MLFlinkCpp
+8
$236,000–$290,000/yr
gross
🏢 On-site
San Francisco
🗣️ EN

Job
· Principal
Staff Software Engineer, AI Tooling
Airwallex
Cpp
+2
$200,000–$280,000/yr
gross
🔀 Hybrid
US - San Francisco
🗣️ EN