AI Product Analyst - Voice AI Agents

Stack

Responsibilities

• ▹Analyze production behavior of Voice AI Agents
• ▹Investigate hallucinations, inconsistencies and silent failures
• ▹Work with call transcripts, logs and generated outputs
• ▹Design evaluation frameworks for measuring AI quality
• ▹Build monitoring systems and dashboards
• ▹Partner with Product, Engineering, QA and Operations teams

Requirements

• ▹3+ years of experience in Product, Data or Operational Analytics
• ▹Strong SQL skills
• ▹Python for analytical work with large text datasets
• ▹Understanding of LLM limitations, hallucinations and evaluation challenges
• ▹Fluent Russian and English at B2 level or higher

Nice to have

• ▹Experience with conversational AI or A/B testing

Soft skills

About the company

MEDvidi is an AI-powered mental healthcare platform combining licensed providers with proprietary AI tools in the United States.