Staff Software Developer
Confluent's Stream Processing & Analytics (SPA) team is building an elastic, reliable, durable, cost-effective and performant stream processing engine based on Apache Flink for Confluent Cloud. This Staff role is critical in defining and executing the long-term vision around a scalable, efficient, resilient and truly cloud-native data stream processor, including autoscaling and limitless state management. The goal is a zero-knob, worry-free experience that lets customers focus on their use cases instead of infrastructure complexity.
Responsibilities
- ▹Own the architecture and technical strategy for a complex engineering system (e.g. a core component of scheduling, deployment, autoscaling or state management) within the Flink engine
- ▹Contribute to the broader technical vision and translate it into actionable, high-impact projects aligned with Confluent Cloud's strategic goals
- ▹Balance short-term implementation efforts against long-term architectural health
- ▹Independently drive open-ended, complicated technical projects from inception through production, within your area and across peer teams
- ▹Identify and initiate high-leverage, high-impact projects and gain stakeholder buy-in
- ▹Significantly simplify code and design complexity to help the team and company move fast
- ▹Serve as the sought-out technical expert for your domain and improve stability, performance, scalability and operational excellence across critical SPA systems
- ▹Mentor and coach junior and mid-level engineers and set up processes and tools that make others successful
Requirements
- ▹BS, MS or PhD in computer science or a related field, or equivalent work experience, typically with 8+ years of relevant experience in stream processing or large-scale distributed systems
- ▹Demonstrated strong fundamentals and extensive experience in the architecture, design and development of a complex engineering system end to end
- ▹Proven ability to lead a group effort or own a deep technical area individually and align small groups on technical direction
- ▹Strong understanding of cloud engineering principles, including building and operating mission-critical systems in public cloud environments
- ▹Proficiency in Go, Java, Scala or Rust with the ability to contribute technically in these languages
Nice to have
- ▹Deep expertise in the design, implementation and optimization of large-scale, fault-tolerant state management systems, distributed storage systems or high-throughput messaging systems
- ▹Recognized, impactful technical contributions to open-source stream processing projects, particularly Apache Flink
- ▹Hands-on experience designing and operating Kubernetes operators for distributed applications
Soft skills
About the company
Confluent builds a data streaming platform based on Apache Kafka that puts information in motion in near real-time. Confluent is an IBM subsidiary that has been acquired by IBM and is being integrated into the IBM organization.