Member of Technical Staff (Data Scientist/Engineer, Online Metrics)

Stack

Responsibilities

• ▹Discover and validate online signals that serve as reliable proxies for true answer quality
• ▹Design and implement new online metrics for A/B testing and product health dashboards, aligned with ground-truth evaluations
• ▹Analyze experimental results to validate metrics, ensuring they accurately predict user satisfaction
• ▹Build and maintain data pipelines that calculate these metrics at scale
• ▹Communicate findings in close collaboration with Product and Search teams
• ▹Operate in a small, high-impact team shaping how Perplexity measures and improves Answer Quality

Requirements

• ▹MS in a technical field or equivalent experience
• ▹4+ years of experience as a Data Scientist, Analytics Engineer, or related role
• ▹Experience with search, recommendation, or LLM-based products, designing online metrics and analyzing A/B experiments
• ▹Strong proficiency in Python and SQL (production-grade code expected)
• ▹Deep knowledge of statistical analysis
• ▹Experience with Business Intelligence (BI) tools for visualization and reporting
• ▹Comfortable with agentic coding workflows and AI-assisted development tools

Nice to have

• ▹Proficiency with Apache Spark and Databricks
• ▹Experience with the development or validation of LLM-as-a-judge systems
• ▹Prior work supporting customer-facing products at scale

Soft skills

About the company

Perplexity serves tens of millions of users daily with reliable, high-quality answers from its LLM-first search engine and specialized data sources; the Answer Quality team measures and improves answer quality.