Freelance Senior Databricks Platform Engineer

Stack

Responsibilities

• ▹Design and implement Lakehouse architectures on Databricks (medallion patterns, Delta Lake, Unity Catalog)
• ▹Build batch and streaming data pipelines using PySpark and Spark SQL, orchestrated via Databricks Workflows/Jobs
• ▹Model data for analytics use cases (star schema for BI, curated datasets for data science)
• ▹Ingest data from APIs, files, databases and SaaS sources into data lakes/lakehouses
• ▹Implement testable transformation logic (data quality checks, deduplication, SCD, CDC, business rules)
• ▹Apply data governance best practices (cataloguing, lineage, RBAC, masking of sensitive data)
• ▹Set up CI/CD and deployment pipelines (Azure DevOps/GitHub) for Databricks assets
• ▹Use Terraform for automated Infrastructure as Code setup and reusability
• ▹Monitor, optimize and troubleshoot pipelines and clusters for performance, robustness and cost efficiency
• ▹Help define best practices, coding standards and reusable components

Requirements

• ▹Proven experience in Databricks platform architecture
• ▹Unity Catalog and governance
• ▹CI/CD and DevOps for Databricks (Enterprise GitHub)
• ▹Terraform / Infrastructure as Code
• ▹Lakehouse architecture
• ▹Data Product Platform enablement

Nice to have

• ▹Knowledge of Knowledge Graphs, ontology and agentic AI is a strong plus

What we offer

• ▹Work in an international group with strong local expertise in Belgium
• ▹Collaborate with cross-functional teams in the international data & AI adesso community
• ▹Flexible freelance setup with the support of a growing adesso Belgium community

About the company

adesso Belgium is an IT start-up under the adesso umbrella with a proven track record across industries, part of the adesso group's Belgian expansion.