Databricks Architect

Stack

Responsibilities

• ▹Design and architect scalable Databricks-based solutions for enterprise clients
• ▹Lead end-to-end cloud data platform implementations across AWS and/or Azure
• ▹Design and build production pipelines using Databricks Workflows and Spark Declarative Pipelines
• ▹Design scalable ETL/ELT pipelines using Spark, PySpark, and Databricks Workflows
• ▹Partner directly with client stakeholders to translate business needs into technical solutions
• ▹Provide technical leadership and guidance to data engineering teams
• ▹Implement CI/CD pipelines and infrastructure-as-code for Databricks resources
• ▹Drive collaborative development through Git workflows, pull requests, and code reviews
• ▹Serve as the primary technical point of contact for clients, including C-level executives

Requirements

• ▹6+ years of experience in data engineering, cloud architecture, or related roles
• ▹3+ years of hands-on experience with Databricks
• ▹Strong proficiency in PySpark, Spark SQL, Python, and SQL
• ▹Deep experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs
• ▹Proven experience delivering Databricks projects in a consulting or professional services environment
• ▹Strong understanding of data lake concepts and formats such as Delta, Iceberg, and Parquet
• ▹Experience designing and managing Unity Catalog for governance, access control, and lineage
• ▹Familiarity with medallion architecture (bronze/silver/gold) patterns
• ▹Hands-on experience with Git version control, pull requests, code reviews, and collaborative development workflows
• ▹CI/CD and infrastructure-as-code experience for Databricks resources (Terraform, Asset Bundles, GitHub Actions)
• ▹MLflow experience: experiment tracking, model registry, deployment, and observability
• ▹Experience working in client-facing, consulting, or cross-functional environments
• ▹Experience leading technical projects or mentoring engineering teams
• ▹Must be located in Colombia (Bogotá, Medellin, Barranquilla, or Cali)

Nice to have

• ▹Databricks certifications (Data Engineer Associate/Professional, Machine Learning Associate/Professional)
• ▹Experience with agentic AI workflows, including Databricks Agentbricks, LangChain, and Claude agentic workflows

Soft skills

What we offer

• ▹Comprehensive prepaid health plan for the employee and immediate family
• ▹Team engagement activities
• ▹Annual performance bonus
• ▹Career growth opportunities
• ▹Flexible hybrid work model, mostly remote with one office day per week
• ▹Paid time off and holidays

About the company

Cuesta Partners is a consulting company delivering data engineering and cloud data platform solutions for enterprise clients.