LLM Red-Teamer

Stack

Responsibilities

• ▹Develop complex, adversarial multi-turn conversations and task-based scenarios aligned with detailed project specifications
• ▹Author clear, precise evaluation rubrics to rigorously assess model responses
• ▹Iteratively test conversations and tasks against frontier LLMs, escalating difficulty and nuance
• ▹Deliver comprehensive task packages, including transcripts, target behaviors, binary rubrics, and supporting rationale
• ▹Validate LLM outputs, documenting model strengths and failure modes
• ▹Maintain calibration with team leads and quality control contacts

Nice to have

• ▹Exceptional written English skills, with clarity, precision, and strong structural organization
• ▹Prior experience in AI human data environments (RLHF, SFT, evaluations, annotation, or prompt engineering)
• ▹Deep familiarity with large language models, including anticipating common failure patterns
• ▹Demonstrated ability to work autonomously with minimal oversight
• ▹Proven critical thinking and meticulous attention to detail
• ▹Experience designing evaluation items or rubrics
• ▹Background in writing-intensive or analysis-centric fields (research, editorial, technical writing, quality assurance)

Soft skills

What we offer

• ▹Output-based compensation, paid per completed task