← Zurück zur Liste
Stelle

Data Platform Engineer

Data Engineer • Remote • Vollzeit • Portugal Portugal

Own the end-to-end real-time pipeline of a new multi-tenant data platform, serving data through a unified analytical warehouse and feature-serving layer, from real-time decisioning to predictive modelling. Fully remote.

Responsibilities

  • ▹Design, build and maintain scalable data pipelines using AWS Glue (PySpark) or equivalent orchestration and transformation tools
  • ▹Engineer and optimise the ClickHouse warehouse for sub-second query performance across all back-offices
  • ▹Implement data contracts between back-office systems and the platform
  • ▹Build the feature-serving layer providing pre-computed features to AI agents at millisecond latency
  • ▹Integrate with third-party databases, back-office APIs and external systems (CRM, affiliates, acquisition platforms)
  • ▹Establish monitoring, alerting and maintenance procedures: pipeline health checks, freshness monitoring, anomaly detection, data contract SLA enforcement
  • ▹Own CI/CD and infrastructure-as-code for data workloads
  • ▹Collaborate with data scientists, agent engineers, BI developers and infrastructure teams

Requirements

  • ▹3+ years building and operating production data pipelines at scale, across streaming and batch
  • ▹Expertise in Apache Kafka (or Amazon MSK): topic design, consumer group management, offset handling, schema registry and troubleshooting lag, rebalancing and throughput
  • ▹Strong SQL and warehouse engineering skills with columnar analytical databases (ClickHouse strongly preferred, or Druid, BigQuery, Redshift)
  • ▹PySpark / Spark Streaming proficiency: transformation jobs that normalise, enrich and enforce business rules on event streams
  • ▹Data modelling discipline: normalised, multi-tenant schemas where tenant isolation is a filter, not a fork
  • ▹Experience with data contracts and schema governance
  • ▹CI/CD and infrastructure-as-code experience: automated testing of data pipelines, version-controlled deployments (CloudFormation, Terraform or CDK), familiarity with containerised workloads (ECS Fargate or Kubernetes)
  • ▹Data quality and observability mindset: pipeline health monitoring, automated data validation (Great Expectations or equivalent), freshness checks, anomaly detection

Nice to have

  • ▹Experience with AWS Glue, Apache Airflow or Apache NiFi
  • ▹Experience in iGaming, online casino, poker or sportsbook platforms
  • ▹Exposure to blockchain or crypto-native transaction flows: on-chain event ingestion, token-denominated accounting, stablecoin settlement
  • ▹Comfortable in an AWS-native environment (MSK, Glue, S3, DynamoDB, ECS, IAM), understanding serverless tradeoffs and cost-efficient sizing
  • ▹Feature store experience (SageMaker, Feast or Tecton) building offline/online feature pipelines for ML inference
  • ▹Prior work in regulated industries (financial services, gambling, fintech) where data lineage, auditability and compliance are non-negotiable
  • ▹Experience migrating legacy query engines (Athena, Trino, Presto) to modern analytical warehouses with reconciliation frameworks

Soft skills

Collaboration across several teamsOwnership of domain architecture decisionsParticipation in design reviews

What we offer

  • ▹Fully remote, asynchronous-first communication
  • ▹Small, high-autonomy team within the Data function
  • ▹Build phase focuses on velocity with quality, no firefighting of legacy systems

About the company

Block Labs is a technology studio working at the edge of Web3, AI and iGaming, building high-scale, production-grade platforms.

Ähnliche Stellen