Distributed Systems Engineer - Data Platform - Logs and Audit Logs

Stack

Responsibilities

• ▹Design, build and operate a robust logging platform, ensuring reliable logging and secure data transfer to a wide array of customer destinations and third-party integrations
• ▹Develop and maintain high-performance data connectors and integrations for log-shipping products, focusing on usability, scalability and data integrity
• ▹Create and manage systems for handling comprehensive audit logs, ensuring secure delivery and adherence to strict compliance and performance standards
• ▹Scale and optimize the data delivery pipeline to handle massive data volumes with low latency, identifying and removing bottlenecks in processing and routing
• ▹Work closely with Product and other engineering teams to define requirements for a new logging platform and integrations
• ▹Maintain operational health of the log delivery platform through comprehensive monitoring and participation in an on-call rotation (with flexibility for out-of-hours issue resolution)
• ▹Collaborate on the architectural evolution of the data egress platform, researching and implementing new technologies to improve efficiency and reliability

Requirements

• ▹3+ years of experience in software development covering distributed systems and data pipelines
• ▹Strong programming skills (Go preferable), with a deep understanding of best practices for building resilient, high-throughput systems
• ▹Hands-on experience with modern observability stacks (Prometheus, Grafana) and handling high-cardinality metrics at scale
• ▹Strong knowledge of SQL, including query optimization
• ▹Solid foundation in computer science: algorithms, data structures, distributed systems and concurrency
• ▹Strong analytical and problem-solving skills, with willingness to debug and troubleshoot complex problems at scale
• ▹Ability to work collaboratively and communicate effectively with other teams across Cloudflare

Nice to have

• ▹Experience with data streaming technologies (e.g. Kafka, Flink)
• ▹Experience with logging platforms or SIEMs (e.g. Splunk, Datadog, Sumo Logic) and storage destinations (e.g. S3, R2, GCS)
• ▹Experience with Infrastructure as Code tools like SALT or Terraform
• ▹Experience with Linux container technologies such as Docker and Kubernetes

Soft skills

What we offer

• ▹Medical/Rx Insurance
• ▹Dental Insurance
• ▹Vision Insurance
• ▹Flexible Spending Accounts
• ▹Commuter Spending Accounts
• ▹Fertility & Family Forming Benefits
• ▹On-demand mental health support and Employee Assistance Program
• ▹Global Travel Medical Insurance

About the company

Cloudflare's Data organization owns the entire data lifecycle — from ingestion and processing to storage and retrieval — handling over a billion events per second to give customers real-time visibility into the health, performance and security of their online properties.