Staff Software Engineer - Distributed Data Systems

Stack

Responsibilities

• ▹Apache Spark: develop the de facto open source standard framework for big data
• ▹Data Plane Storage: deliver reliable, high-performance services and client libraries for storing and accessing huge amounts of data on cloud storage backends (AWS S3, Azure Blob Store)
• ▹Delta Lake: a storage management system combining the scale and cost-efficiency of data lakes, the performance and reliability of a data warehouse, and the low latency of streaming, with ACID transactions and time travel
• ▹Delta Pipelines: make it simple to orchestrate and operate tens of thousands of data pipelines
• ▹Performance Engineering: build the next-generation query optimizer and execution engine that's fast, tuning-free, scalable, and robust

Requirements

• ▹BS in Computer Science, a related technical field, or equivalent practical experience
• ▹Comfortable working towards a multi-year vision with incremental deliverables
• ▹Driven by delivering customer value and impact
• ▹8+ years of production-level experience in Java, Scala, or C++
• ▹Strong foundation in algorithms and data structures and their real-world use cases
• ▹Experience with distributed systems, databases, and big data systems (Apache Spark, Hadoop)

Nice to have

• ▹MS or PhD in databases or distributed systems (optional)

Soft skills

About the company

Databricks is the data and AI company, founded in 2013 by the original creators of Apache Spark. From a small office in Berkeley, it has grown into a global organization with over 1000 employees. More than 10,000 organizations, including over 50% of the Fortune 500, rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI.