Data Science Expert - AI Evaluation

Stack

Responsibilities

• ▹Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards and experiment readouts
• ▹Score AI-generated and human work samples against criteria with detailed written justifications
• ▹Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible
• ▹Incorporate structured feedback from senior reviewers and iterate quickly
• ▹Work independently and asynchronously to meet deadlines

Requirements

• ▹5+ years of professional data science experience in industry
• ▹Background in business operations, product or growth data science at top-tier technology companies
• ▹Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis and communicating findings to executive stakeholders
• ▹Exceptionally strong written communication
• ▹Detail-oriented, consistent, comfortable having judgment reviewed and calibrated against peers

Nice to have

• ▹Prior experience with AI training, evaluation or human-data projects

Soft skills

What we offer

• ▹$120-170/hour
• ▹Fully remote

About the company

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, its investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers and Jack Dorsey.