Senior Research Engineer (Agentic Behavior)

Stack

Responsibilities

• ▹Design and implement tooling to systematically capture, classify, and analyse errors that AI coding agents make when generating Kotlin code
• ▹Build observability pipelines over agentic traces, mining patterns from agent sessions in JetBrains IDEs, Junie, Claude Code, Cursor, and other coding agents
• ▹Design, implement, and maintain evaluation pipelines that measure Kotlin code generation quality across correctness, idiomaticity, build success, framework usage, and test coverage
• ▹Build simulation environments to measure coding agents on realistic Kotlin developer tasks, from greenfield KMP projects to migrating Spring applications from Java to Kotlin
• ▹Own evaluation infrastructure: metrics, experiment tracking, automated regression checks, and reproducible benchmarking
• ▹Experiment with post-training techniques (SFT, DPO, GRPO) to improve how models handle Kotlin-specific patterns, idioms, and frameworks
• ▹Investigate context engineering approaches: CLAUDE.md/AGENTS.md files, compiler-as-verifier feedback loops, Kotlin LSP integration, and MCP-based tooling
• ▹Run experiments to measure impact through A/B comparisons, benchmark suites, and before/after analyses on real codebases
• ▹Collaborate with model providers (Anthropic, OpenAI, and Google) to translate Kotlin-specific findings into model improvements
• ▹Design and build open-source benchmarks that measure AI coding agent performance on Kotlin tasks
• ▹Create task datasets covering the breadth of Kotlin usage: server side (Spring, Ktor), multiplatform projects (KMP), build systems (Gradle), and Android

About the company

At JetBrains, code has been the team's passion since 2000, striving to build the strongest, most effective developer tools. The Kotlin AI Value Stream team is responsible for how AI agents understand, generate, and improve Kotlin code across Android, Kotlin Multiplatform, server-side, web, desktop and other platforms.