Senior Data Engineer (Python / AWS / ML Pipelines)

Stack

Responsibilities

• ▹Build and maintain data and machine learning pipelines in production
• ▹Develop ETL and data processing workflows using Python
• ▹Orchestrate workflows using tools such as Airflow and AWS Step Functions
• ▹Work with AWS Glue for data processing and pipeline execution
• ▹Support the deployment of machine learning models into production using SageMaker
• ▹Work closely with Data Scientists and ML Engineers to productionise ML models
• ▹Monitor, troubleshoot and improve pipeline reliability and performance
• ▹Contribute to architecture and technical decisions around the data platform and ML infrastructure
• ▹Work in an agile environment with distributed teams

Requirements

• ▹Strong experience with Python
• ▹Experience building data pipelines and/or ML pipelines in production
• ▹Hands-on experience with Airflow
• ▹Experience with AWS Step Functions and AWS Glue
• ▹Experience deploying ML models using AWS SageMaker
• ▹Strong experience with AWS cloud
• ▹Experience working with production systems at scale
• ▹Experience working in cross-functional teams with Data Scientists
• ▹Strong communication skills and ability to work in a distributed team environment
• ▹Strong English communication skills, both written and verbal
• ▹This position is for Portugal or Brazil based candidates only

Nice to have

• ▹Experience with GCP
• ▹Experience with monitoring and observability tools

Soft skills

What we offer

• ▹A collegial environment where responsibility and authority are shared equally amongst colleagues
• ▹A culture that doesn't criticise failure but learns from mistakes
• ▹An agile environment where ideas are welcome
• ▹The possibility to grow and experience different projects
• ▹Ongoing training and mentoring
• ▹The possibility of travel