LLMOps Engineer

Stack

Responsibilities

• ▹Build and run eval suites per product, decomposed by failure mode, running on schedule and on every release
• ▹Keep evals cost-effective as the product line grows
• ▹Own the reporting loop: put eval and platform data findings in front of the team and stakeholders
• ▹Surface unintended or problematic model behaviour before learners do
• ▹Steward the core datasets the team depends on, including classified customer-support data
• ▹Partner with the AI Product Manager on instrumentation

Requirements

• ▹Python & data: solid Python and a data inclination, comfortable shaping and analysing messy LLM-generated data
• ▹Decomposition: the ability to look at an AI product and decompose it into success and failure metrics
• ▹Familiarity with the eval landscape

About the company

ALX Africa, a non-profit organisation under the ALX Foundation, is dedicated to unlocking the potential of Africa's digital future. It provides world-class tech skills training and career acceleration programmes, aiming to empower 2 million Africans into sustainable tech careers by 2030. It has hubs in 8 cities across Africa, with 347k+ graduates and 257k youth in work so far.