Member of Technical Staff (Search Quality Analyst)

Stack

Responsibilities

• ▹Find and diagnose quality issues in the search pipeline
• ▹Design metrics from scratch to track and measure search quality
• ▹Build datasets for model training, including LLM-as-a-judge labeling pipelines
• ▹Improve search snippet quality and page selection algorithms for indexing
• ▹Design and analyze A/B experiments to validate improvements

Requirements

• ▹4+ years of experience as a data analyst or in a related role
• ▹Strong coding skills: production-grade code at a mid-level backend engineer level
• ▹Proficiency with SQL and Python
• ▹Demonstrated hands-on experience with at least one of: designing metrics from scratch; building labeling pipelines using LLM-as-a-judge; training ML models that shipped to production with measurable metric improvements; designing evals with known ground truth (e.g. SimpleQA, BrowseComp) or driving meaningful improvements on them

Nice to have

• ▹Experience working on search-related products
• ▹Knowledge of statistics and A/B experiment design
• ▹Experience with Apache Spark or Databricks

About the company

Perplexity builds AI-powered search and answer systems.