Own the end-to-end real-time pipeline of a new multi-tenant data platform, serving data through a unified analytical warehouse and feature-serving layer, from real-time decisioning to predictive modelling. Fully remote.
Stack
Responsibilities
- ▹Design, build and maintain scalable data pipelines using AWS Glue (PySpark) or equivalent orchestration and transformation tools
- ▹Engineer and optimise the ClickHouse warehouse for sub-second query performance across all back-offices
- ▹Implement data contracts between back-office systems and the platform
- ▹Build the feature-serving layer providing pre-computed features to AI agents at millisecond latency
- ▹Integrate with third-party databases, back-office APIs and external systems (CRM, affiliates, acquisition platforms)
- ▹Establish monitoring, alerting and maintenance procedures: pipeline health checks, freshness monitoring, anomaly detection, data contract SLA enforcement
- ▹Own CI/CD and infrastructure-as-code for data workloads
- ▹Collaborate with data scientists, agent engineers, BI developers and infrastructure teams
Requirements
- ▹3+ years building and operating production data pipelines at scale, across streaming and batch
- ▹Expertise in Apache Kafka (or Amazon MSK): topic design, consumer group management, offset handling, schema registry and troubleshooting lag, rebalancing and throughput
- ▹Strong SQL and warehouse engineering skills with columnar analytical databases (ClickHouse strongly preferred, or Druid, BigQuery, Redshift)
- ▹PySpark / Spark Streaming proficiency: transformation jobs that normalise, enrich and enforce business rules on event streams
- ▹Data modelling discipline: normalised, multi-tenant schemas where tenant isolation is a filter, not a fork
- ▹Experience with data contracts and schema governance
- ▹CI/CD and infrastructure-as-code experience: automated testing of data pipelines, version-controlled deployments (CloudFormation, Terraform or CDK), familiarity with containerised workloads (ECS Fargate or Kubernetes)
- ▹Data quality and observability mindset: pipeline health monitoring, automated data validation (Great Expectations or equivalent), freshness checks, anomaly detection
Nice to have
- ▹Experience with AWS Glue, Apache Airflow or Apache NiFi
- ▹Experience in iGaming, online casino, poker or sportsbook platforms
- ▹Exposure to blockchain or crypto-native transaction flows: on-chain event ingestion, token-denominated accounting, stablecoin settlement
- ▹Comfortable in an AWS-native environment (MSK, Glue, S3, DynamoDB, ECS, IAM), understanding serverless tradeoffs and cost-efficient sizing
- ▹Feature store experience (SageMaker, Feast or Tecton) building offline/online feature pipelines for ML inference
- ▹Prior work in regulated industries (financial services, gambling, fintech) where data lineage, auditability and compliance are non-negotiable
- ▹Experience migrating legacy query engines (Athena, Trino, Presto) to modern analytical warehouses with reconciliation frameworks
Soft skills
Collaboration across several teamsOwnership of domain architecture decisionsParticipation in design reviews
What we offer
- ▹Fully remote, asynchronous-first communication
- ▹Small, high-autonomy team within the Data function
- ▹Build phase focuses on velocity with quality, no firefighting of legacy systems
About the company
Block Labs is a technology studio working at the edge of Web3, AI and iGaming, building high-scale, production-grade platforms.
Similar jobs
Job
Data Engineer
Entain
Azure DevopsBigquery
+12
💰 Salary: not specified
🏢 On-site
Lisbon
🗣️ EN

Job
Infrastructure & Data Engineer
BGBx
BigqueryDagsterDatabricksDatadog
+13
€55,000/yr
gross
🏢 On-site
🗣️ EN

Job
Data Engineer GCP (Google Cloud Plataform)
act digital
Bigquery
+2
💰 Salary: not specified
🏢 On-site
Porto
🗣️ EN

Job
Data Engineer
GoCardless
AI/MLBigquery
+7
€58,400–€87,600/yr
gross
🏢 On-site
Lisbon
🗣️ EN

Job
Data Engineer, Portugal
Zego
AI/MLData ScienceDbt
+8
€35,000–€50,000/yr
gross
🔀 Hybrid
🗣️ EN

Job
Data Engineer
Patrianna
FlinkBigqueryClickhouse
+11
💰 Salary: not specified
🔀 Hybrid
Gibraltar
🗣️ EN
