Staff Software Engineer, Data Platform

Stack

Responsibilities

• ▹design scalable data pipelines for ingesting telemetry, logs and sensor data
• ▹develop and maintain data ingestion agents on robot software in low-connectivity environments
• ▹develop data formats and protocols optimized for storage, transmission and query
• ▹build observability systems (fluentbit, OpenTelemetry) for real-time monitoring
• ▹implement big data processing solutions using Spark, Kafka or equivalent
• ▹build internal BI and long-horizon fleet analytics tools on AWS (S3, Athena, Sagemaker)

Requirements

• ▹5+ years building and scaling cloud-based data platforms (AWS, GCP or Azure)
• ▹proficiency with big data frameworks (Spark, Kafka, Beam, Flink)
• ▹familiarity with open data formats (Parquet, Arrow, Iceberg) and protocols (Avro, Protobuf)
• ▹experience with observability tools (OpenTelemetry, Prometheus)
• ▹proficiency in Java, Scala or Python, and distributed systems/microservices

Nice to have

• ▹experience with robotics, autonomous vehicles or sensor-heavy workloads
• ▹knowledge of IoT systems and edge computing
• ▹data catalog experience (Datahub, Amundsen, Databricks Unity Catalog)
• ▹experience with Databricks, Kubernetes, Bazel/Blaze
• ▹experience with C++, Rust
• ▹familiarity with robot data formats (mcap, ROS) and playback/visualization tooling

What we offer

• ▹fully remote with hybrid option from Salem OR, Pittsburgh PA or Fremont CA offices

About the company

Agility Robotics operates commercially deployed humanoid robots in warehouses, manufacturing and distribution centers, with industry-leading safety standards.