Domain Advisor Consultant (Subject Matter Expert)

Stack

Responsibilities

• ▹Rubric Architecture: review and advise on quality criteria used to evaluate AI outputs
• ▹Golden data curation: create, edit, or certify benchmark Golden Tasks
• ▹Expert gap analysis: analyze model failures and explain how a human expert would approach the problem
• ▹Advisory on complexity: consult on the difficulty and nuance of prompts

Requirements

• ▹Elite domain mastery: advanced degrees (PhD, MD, JD) or equivalent industry-recognized status (e.g., Senior Principal Engineer, published author, award-winning creative)
• ▹Operates as an independent consultant or freelancer with own workflows and methodology
• ▹Ability to articulate reasoning clearly in writing
• ▹Properly registered as an independent contractor in country of residence

Nice to have

• ▹Prior experience in AI evaluation

Soft skills

What we offer

• ▹Flexible, non-exclusive, project-based engagements alongside other clients
• ▹Flexibility in how deliverables are completed
• ▹Scope, timelines, and compensation defined in advance
• ▹Remote work with own workspace