Senior/Staff Machine Learning Engineer - Health Evaluation - AI Teams (x/f/m)
Doctolib is on a mission to transform healthcare delivery with AI. As a Senior/Staff Machine Learning Engineer, you will play a key role in designing, implementing and scaling the evaluation framework that ensures the AI Health Companion behaves safely, reliably and helpfully for millions of patients and practitioners. You will join a cross-functional team of ML engineers, product engineers and medical experts to build robust evaluation pipelines for agentic AI systems capable of reasoning, planning and interacting with complex healthcare data. The position is permanent, full-time, hybrid in the Levallois office.
Responsibilities
- ▹Define and own the evaluation strategy for the AI agentic system: metrics, protocols, datasets and tooling
- ▹Implement and maintain automated evaluation pipelines to monitor model quality, safety and alignment across iterations
- ▹Run systematic experiments to assess reasoning, factuality, robustness and user experience
- ▹Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement
- ▹Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices
Requirements
- ▹MSc or PhD in Computer Science, Machine Learning, Data Science or a related field
- ▹7+ years of hands-on experience with large language models (e.g., GPT, Claude, Llama or BERT-like architectures)
- ▹Proven experience evaluating agentic or reasoning systems (autonomous agents, tool-using LLMs, dialogue systems or task-oriented assistants)
- ▹Strong track record in experiment design, metric definition and evaluation automation
- ▹Ability to bridge research and production, influencing modeling and product decisions
- ▹Excellent communication skills and a collaborative mindset
Nice to have
- ▹Experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI
Soft skills
What we offer
- ▹Free health insurance for you and your children
- ▹Parent Care Program: one additional month of leave on top of legal parental leave
- ▹Free mental health and coaching services through Moka.care
- ▹Package for caregivers and workers with disabilities (adapted remote policy, extra days off for medical reasons, psychological support)
- ▹Work from EU countries and the UK for up to 10 days per year (flexibility days policy)
- ▹Work council subsidy for sport club membership or creative classes
- ▹Up to 14 days of RTT
- ▹Lunch voucher with Swile card
About the company
Doctolib's solutions run on a single fully cloud-native platform supporting web and mobile interfaces in multiple languages, adapted to country and healthcare specialty requirements; the platform is being modularized into a distributed architecture with reusable components. The stack includes Rails, TypeScript, Java, Python, Kotlin, Swift and React Native, and AI is used ethically to empower patients and health professionals.