LLMOps Engineer

Stack

Responsibilities

• ▹Build and run eval suites per product, decomposed by failure mode, running on schedule and on every release
• ▹Keep evals cost-effective as the product line grows
• ▹Own the reporting loop, surfacing findings from evals and platform data to the team and stakeholders
• ▹Steward the core datasets the team depends on, including classified customer-support data
• ▹Partner with the AI Product Manager on instrumentation

Requirements

• ▹Solid Python skills and a data inclination, comfortable shaping and analysing messy LLM-generated data
• ▹Ability to decompose an AI product into distinct, testable failure modes

About the company

ALX Africa is a non-profit organisation under the ALX Foundation dedicated to unlocking Africa's digital future through world-class tech skills training and career acceleration programmes. It aims to empower 2 million Africans into sustainable tech careers by 2030, operates hubs in 8 African cities, and has produced 347k+ graduates since 2020.