Member of Engineering (Evaluations / Engineering)

Responsibilities

• ▹Design a Python framework that makes it easy to implement both internal and public benchmarks in a centralized way
• ▹Build and maintain the pipeline that runs distributed evaluations at scale
• ▹Collaborate with modeling and product teams to identify opportunities to improve experimentation and evaluation tooling

Requirements

• ▹Strong engineering background
• ▹Experience leading software projects cross-functionally
• ▹Experience building highly reliable and well-tested services
• ▹Experience with distributed systems
• ▹Data pipelines, distributed processing
• ▹Message queues, event-driven architectures, e.g. Kafka, Google Pub/Sub
• ▹Clouds: managed services, storage - GCP, AWS, Azure
• ▹Monitoring and alerting: Grafana, Prometheus, Datadog

Nice to have

• ▹Experience designing frameworks or tooling for developers
• ▹A product mindset toward building developer-facing software
• ▹A knack for collaborating with researchers and ML engineers to identify opportunities
• ▹Experience with ML ops and data visualization platforms

Soft skills

What we offer

• ▹Fully remote work & flexible hours
• ▹37 days/year of vacation & holidays
• ▹Health insurance allowance for you and dependents
• ▹Company-provided equipment
• ▹Wellbeing, always-be-learning and home office allowances
• ▹Frequent team get-togethers
• ▹Great diverse & inclusive people-first culture

About the company

Poolside is building toward Artificial General Intelligence by accelerating software development itself, reshaping the developer experience with agentic systems, coding assistants, and the frontier models that power them. The team works remotely across Europe and North America, gathering in person for 3 days each month plus longer annual off-sites.