Senior Azure Databricks Engineer

Stack

Responsibilities

• ▹Implement and maintain large-scale data pipelines and the Azure/Databricks data platform in close collaboration with the Architect
• ▹Build robust, scalable pipelines using Azure Databricks (PySpark/SQL), integrating data ingestion, processing, transformation, and storage
• ▹Design and enforce data architecture standards: Delta Lake/Lakehouse architecture, data modeling (dimensional models, star/snowflake schemas)
• ▹Manage ingestion and orchestration using Azure Data Services (ADF, ADLS, Synapse Analytics) for storage, compute, and analytics
• ▹Establish and enforce CI/CD pipelines for data workflows and infrastructure using Azure DevOps or equivalent tools
• ▹Implement and uphold data governance, metadata management, and data quality practices
• ▹Monitor, troubleshoot, and optimize performance of data processing jobs (Spark/Databricks)
• ▹Collaborate with cross-functional teams to understand requirements and explain complex technical concepts to non-technical audiences
• ▹Proactively research, evaluate, and adopt new data/cloud technologies and best practices

Requirements

• ▹5+ years of experience as a data engineer (or similar), with significant exposure to cloud-based data platforms and modern data architectures
• ▹Proven hands-on experience building and managing data pipelines using Azure Databricks (PySpark + SQL) in production environments
• ▹Strong understanding of data modeling, data warehousing/lakehouse architecture (dimensional modeling, star/snowflake schemas, Delta Lake/Lakehouse)
• ▹Proven track record in data governance, metadata management, and ensuring data quality at scale
• ▹Solid experience in performance optimization and troubleshooting of data processing jobs/pipelines (Spark/Databricks)
• ▹Excellent programming and query skills - strong SQL, Python (or another relevant language)
• ▹Strong analytical and problem-solving skills at both micro (pipeline/job) and macro (architecture/strategy) levels
• ▹Excellent communication skills: explaining complex technical issues to non-technical stakeholders, collaborating across teams
• ▹Self-motivated, responsible, capable of working independently and as a technical leader

Nice to have

• ▹Prior experience working in regulated domains (e.g. Finance, Asset Management, Pensions)
• ▹Previous involvement in platform modernization or cloud migration projects
• ▹Familiarity with additional big data/data-engineering tools and patterns beyond Spark/Databricks - streaming, real-time data ingestion, advanced orchestration, metadata tooling, monitoring & alerting
• ▹Experience mentoring or leading smaller data engineering teams/peers
• ▹Familiarity with CI/CD for data workflows, preferably with Azure DevOps, including infrastructure-as-code (Terraform/ARM templates)
• ▹Experience with Azure Data Services such as ADF, ADLS, Synapse (or comparable cloud data services)
• ▹Employ infrastructure-as-code (Terraform or ARM templates) to provision, manage, and version cloud infrastructure and data platform resources

Soft skills

What we offer

• ▹Participation in full-cycle and diverse international projects
• ▹Opportunity to work as a specialist or manager/team lead
• ▹Constant career development with internal mentorship
• ▹Access to learning platforms, paid certifications
• ▹Flexible working conditions, home working allowed
• ▹Attractive base salary and wide range of benefits including cafeteria, bonuses, private health insurance, life insurance, AYCM sport card, referral bonus, family-oriented benefits, discounted company shares

About the company

Accenture is a leading global professional services company in strategy and consulting, interactive marketing, IT, and business operations, with more than 790,000 employees serving clients in over 120 countries.