Databricks Architect | US

Stack

Responsibilities

• ▹Design and architect scalable Databricks-based solutions for enterprise clients
• ▹Lead end-to-end cloud data platform implementations across AWS and/or Azure
• ▹Design and build production pipelines using Databricks Workflows and Spark Declarative Pipelines
• ▹Design scalable ETL/ELT pipelines using Spark, PySpark, and Databricks Workflows
• ▹Partner directly with client stakeholders to translate business needs into technical solutions
• ▹Provide technical leadership and guidance to data engineering teams
• ▹Implement CI/CD pipelines and infrastructure-as-code for Databricks resources
• ▹Drive collaborative development through Git workflows, pull requests, and code reviews
• ▹Collaborate cross-functionally with analytics, engineering, and business teams
• ▹Serve as the primary technical point of contact for clients, including C-level executives

Requirements

• ▹6+ years of experience in Data Engineering, Cloud Architecture, or related roles
• ▹3+ years of hands-on experience with Databricks
• ▹Strong proficiency in PySpark, Spark SQL, Python, and SQL
• ▹Deep experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs
• ▹Proven experience delivering Databricks projects in a consulting or professional services environment
• ▹Strong understanding of data lake concepts and formats (Delta, Iceberg, Parquet)
• ▹Experience designing and managing Unity Catalog for governance, access control, and lineage
• ▹Familiarity with medallion architecture (bronze/silver/gold) patterns
• ▹Hands-on experience with Git version control, pull requests, code reviews, collaborative workflows
• ▹CI/CD and infrastructure-as-code for Databricks resources (Terraform, Asset Bundles, GitHub Actions)
• ▹MLflow experience: experiment tracking, model registry, deployment, and observability
• ▹Cloud platform experience
• ▹Experience working in client-facing, consulting, or cross-functional environments
• ▹Experience leading technical projects or mentoring engineering teams

Nice to have

• ▹Databricks certifications (Data Engineer Associate/Professional, Machine Learning Associate/Professional, ML Engineer Associate/Professional)
• ▹Experience with agentic AI workflows on Databricks (Agentbricks), as well as LangChain and Claude agentic workflows

Soft skills

What we offer

• ▹Constant opportunities for exposure and learning
• ▹Flexible working location, work-life harmony
• ▹Agency and influence in the company's total strategy and direction
• ▹Collaboration with a high-performing team
• ▹Competitive base salary and target bonus of 20-25%
• ▹401k, healthcare benefits, paid time-off, and more

About the company

Cuesta Partners is a consulting firm leading the delivery of data engineering solutions in client-facing environments, often working directly with C-level stakeholders.