← Zurück zur Liste
Stelle · Senior

Spark Data Engineer

Data Engineer • Senior • Remote • Vollzeit • Spanien Spanien

SDG Group's Data Technologies team is hiring a Spark Data Engineer to design and build end-to-end big data solutions in cloud environments.

Responsibilities

  • ▹Design and develop end-to-end solutions for processing large volumes of data, from business problem to scalable ETL/ELT pipelines
  • ▹Work with technologies such as Apache Spark, Python, Scala, Apache Beam, and orchestration with Airflow or Databricks Workflows
  • ▹Process and store data using formats such as Parquet, Delta Lake and Apache Iceberg in cloud environments (AWS, Azure, GCP)
  • ▹Define advanced architectures such as Data Lakehouse, Data Mesh and data warehouses
  • ▹Collaborate with multidisciplinary teams and stakeholders, turning complex requirements into efficient technical solutions
  • ▹Explore and evaluate emerging technologies such as Databricks, Delta Live Tables, dbt Cloud, Apache Iceberg, Snowflake
  • ▹Drive data quality and observability by applying DataOps, Data Quality and CI/CD

Requirements

  • ▹Degree in Computer Engineering, Telecommunications, Mathematics, Physics or similar
  • ▹6+ years of Big Data experience with distributed processing and ETL/ELT pipelines in complex environments, using Python or Scala, Apache Spark, and platforms like Airflow, Databricks or Snowflake
  • ▹Knowledge of data architectures (Data Lakehouse, cloud) and formats/storage such as Parquet, Delta Lake, Iceberg or Hudi
  • ▹Good engineering practices: Git, CI/CD, DataOps, pipeline monitoring, data quality control and data versioning
  • ▹Innovation mindset, curiosity about emerging technologies
  • ▹Communicative English to work in global environments and collaborate with international teams

Nice to have

  • ▹Experience in real-time streaming projects using Kafka, Kinesis, Flink or Spark Structured Streaming
  • ▹Advanced knowledge of orchestration, data DevOps, governance, quality and security, Data Mesh and distributed architectures
  • ▹Experience in ML pipelines and MLOps integrated with Big Data

Soft skills

Innovative, experimental mindsetCollaboration in multidisciplinary, international teams

About the company

SDG Group is a global leader in Big Data, Business Intelligence, Analytics and Artificial Intelligence; since 1994 it has supported half of the IBEX-35 and other international companies in their digital transformation, through strategic partnerships with Microsoft, Google and AWS.

Languages: Angol: kommunikációképes (nemzetközi csapatokhoz)
Education: Informatika, távközlés, matematika, fizika vagy hasonló területen szerzett felsőfokú végzettség

Ähnliche Stellen