Állás · Senior

Senior Data Engineer

Data Engineer • Senior • Remote • Teljes munkaidő • Európai Unió EU/EMEA
About the Role Join a data infrastructure team building a platform that turns information from multiple sources into reliable, up-to-date company intelligence. You will own the platform end-to-end, from architecture and ingestion through production operations, helping power downstream products with trusted data. What You'll Do Design and build a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows. Develop intake pipelines for provider APIs, webhooks, files, and web scraping, with rate limiting and robust failure handling. Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect. Build event-driven systems with messaging or task queues, including retries, dead-letter queues, and controlled replay. Preserve original payloads in Google Cloud Storage and transform data with dbt into raw, standardized, and curated layers. Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures. Design and optimize PostgreSQL schemas and loading patterns, and integrate enrichment providers and LLM-based structured extraction. What We're Looking For At least 5 years building and operating production data platforms, with ownership spanning architecture, implementation, deployment, and ongoing operations. Production experience with GCP and dbt, plus workflow orchestration such as Airflow, Dagster, Prefect, or Temporal, including dependency management, retries, and recovery. Strong Python skills, including asynchronous HTTP and concurrent processing with tools such as asyncio, aiohttp, or HTTPX. Strong SQL and PostgreSQL skills in data modeling, query optimization, and reliable incremental loading. Experience with event-driven systems, queues, data contracts, raw and curated storage, retention policies, automated testing, and pipeline monitoring. Experience in a startup or small engineering team is useful. Web scraping, entity resolution, company or people data enrichment, and LLM-based extraction are also relevant. Compensation & Benefits Annual base salary: $80,000 to $120,000 USD. Visa sponsorship is not available. Location Fully remote, candidates must be based in Latin America.

Hasonló állások