Senior Software Engineer - Distributed Data Systems

Stack

Responsibilities

• ▹Develop Apache Spark, the de facto open source standard framework for big data
• ▹Provide reliable, high-performance services and client libraries for storing and accessing huge amounts of data on cloud storage backends such as AWS S3 and Azure Blob Store (Data Plane Storage)
• ▹Build Delta Lake, a storage management system combining data lake scale/cost-efficiency with data warehouse performance/reliability and streaming low latency, including ACID transactions and time travel
• ▹Work on Delta Pipelines, making it simple to orchestrate and operate tens of thousands of data pipelines
• ▹Build the next-generation query optimizer and execution engine that is fast, tuning-free, scalable and robust (Performance Engineering)

Requirements

• ▹BS (or higher) in Computer Science, a related technical field, or equivalent practical experience
• ▹Comfortable working toward a multi-year vision with incremental deliverables
• ▹Motivated by delivering customer value and impact
• ▹5+ years of production-level experience in Java, Scala or C++
• ▹Strong foundation in algorithms and data structures with real-world use cases
• ▹Experience with distributed systems, databases and big data systems (Apache Spark, Hadoop)

Soft skills

What we offer

• ▹Eligibility for annual performance bonus and equity
• ▹Local pay range: $166,000–$225,000
• ▹Comprehensive benefits package (details vary by region)

About the company

Databricks is the data and AI company. More than 10,000 organizations worldwide — including over half of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Founded by the original creators of Apache Spark, Delta Lake and MLflow.