Senior Databricks Engineer / Architect

Stack

Responsibilities

• ▹Work hands-on on a productive enterprise data platform built on Azure Databricks and develop it continuously with the client team
• ▹Develop and optimize data pipelines, data products and central platform components with Apache Spark, PySpark and SQL
• ▹Design modern data lakehouse architectures using Delta Lake, Unity Catalog and Databricks Workflows
• ▹Take independent responsibility for platform components and data products, from technical design through implementation to stable production operation
• ▹Automation, reproducible deployments and clean development processes: CI/CD and Infrastructure as Code, ideally with Terraform
• ▹Integrate long-term into an existing enterprise team and work closely with engineers, architects and business departments

Requirements

• ▹Several years of hands-on experience with Azure Databricks, including in production environments
• ▹Experience building, evolving or operating large, professionally run data platforms
• ▹Sound knowledge in several of the following: Azure Databricks, Apache Spark and PySpark, SQL, Delta Lake, Unity Catalog, Databricks Workflows, data lakehouse architectures, data engineering and data platform engineering, CI/CD and automated deployment, Infrastructure as Code (ideally Terraform)
• ▹Hands-on mentality: architecture means putting decisions into practice and understanding their impact in production
• ▹Independent way of working, early recognition of technical challenges and ownership
• ▹Senior engineer or architect experience, able to make, communicate and implement sound technical decisions
• ▹Very good German and English (C1)

Soft skills

About the company

Woodmark Consulting GmbH is one of the leading German IT consultancies for AI, data and analytics, driven by a passion for data and innovation.