Senior ML Research Scientist, Pegasus

Who we are Video is 90% of the world's data. Most of it is invisible to machines. TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound, and motion — and power production-scale AI workloads across media, entertainment, sports, security, and government. We have raised more than $210 million from NEA, Radical Ventures, Amazon, NVIDIA, Snowflake, Databricks, Index Ventures, NAVER Ventures, Korea Investment Partners, Quadrille Capital, Red Bull Ventures, and AI pioneers including Fei-Fei Li, Silvio Savarese, and Alexandr Wang. We are a global company, headquartered in San Francisco with offices in Seoul, New York, and London, and employees around the world. We believe the differences in our cultural, educational, and life experiences make our products stronger. Building technology that understands the world in all its complexity requires people who see it from every angle. We are looking for individuals who are driven by hard problems and want their work to matter. Come build it with us! About Pegasus Pegasus is TwelveLabs' core video understanding product, turning video into useful analysis by reasoning over visuals, speech, audio, and on-screen text. The team is not building a generic Video LLM in isolation; we build customer-facing video intelligence workflows that require temporal understanding, structured outputs, and production-grade reliability. A key example is Segment , our time-based metadata capability. Instead of asking the model a broad question about a video, customers define the exact segment types they care about and the metadata fields they want back. Pegasus then finds the relevant start and end times and returns structured metadata for each segment, such as titles, summaries, topics, people, visual subjects, confidence, or domain-specific labels. This is designed for workflows where “what happened” is not enough; customers need to know when it happened and receive metadata that can flow directly into search, archive, editing, compliance, or content management systems. For example, a news archive customer can define a segment type like editorial_narratives and ask Pegasus to split a long broadcast into individual stories. For each story, Pegasus can return a timestamped segment with fields such as segment_title , description , editorial_subjects , visual_subjects , names , and confidence . The output is not just a summary of the full video; it is a structured timeline of the video, aligned to the customer's schema. This is the distinction that matters for Pegasus: general video analysis answers questions about video, while Segment turns video into time-based, structured data tailored to a specific business workflow. Learn more about Pegasus! Building Structured Video Assets: A Time-Based Metadata (TBM) Pipeline Quick Shorts demo: YouTube Shorts About the Team The Pegasus team sits at the core of TwelveLabs' video understanding capabilities and is responsible for driving Pegasus, our Video Analysis product. Our focus is on developing multimodal video analysis systems that are designed for high instruction following capability and producing highly complex, hierarchically structured outputs. We focus on shipping products with real-world value rather than doing research in isolation, and we work in a goal-oriented, cross-functional team that encompasses both ML researchers and engineers. Our work covers a broad range of challenges: large-scale distributed training of multi-modal LLMs that span from pre-training to RL, accurate temporal segmentation and structured metadata extraction for real-world use cases, extending temporal context length to multiple hours, and data curation processes that enable well-aligned evaluation and performance improvements through training data enhancements. Our team has access to the most advanced chips in the world, including NVIDIA B300s, to push the boundaries of video analysis systems—accelerating our research-to-production cycle as fast as possible. In this role, you will Drive research on Pegasus's harder problems such as temporal segmentation, multi-hour context, structured output generation, and training strategies from pre-training through RL, where the right approach requires deep judgment. Design rigorous experiments and evaluation methods that produce clear signals on complex multimodal problems, including where ground truth is ambiguous. Strengthen the team's research approach by helping reframe problems, sharpen hypotheses, and raise the bar for experimental rigor. Work closely with ML Engineers to translate research advances into production, informing tradeoffs around architecture, serving, and system design. Communicate research findings clearly and use them to inform technical direction across the team. Explore and adopt AI-assisted development tools such as Claude, Gemini, and GPT to improve productivity across coding, experimentation, debugging, and documentation. Even if you don't check every box, we encourage you to apply. If you're a zero-to-one achiever, a ferocious learner, and a kind team player who motivates others, you'll find a home at TwelveLabs. You may be a good fit if you have Significant research experience in one or more areas relevant to video understanding, such as multimodal LLMs, large-scale distributed training, temporal modeling, data-centric model development, computer vision, or vision-language systems, with demonstrated depth in at least one. A track record of driving research on problems with significant technical ambiguity, demonstrated through projects, publications, or technical contributions. Strong proficiency in Python and PyTorch. Exceptional experimental judgment, including the ability to design evaluations for complex multimodal problems, run rigorous ablations, and draw clear conclusions from empirical results. Strong communication skills and a track record of strengthening others' research through collaboration — helping formulate sharper hypotheses, identify more informative experiments, or reframe problems more tractably. Preferred qualifications Experience working on multimodal systems involving video, vision, language, or structured output generation. Experience improving model quality through data curation, evaluation design, or training data enhancements. Experience with large-scale distributed training in high-performance GPU environments. Experience translating research advances into production ML systems. Experience defining research direction within a team or project. MS, PhD, or equivalent practical experience in Machine Learning, Computer Science, or a related technical field. Others Work Location: Seoul Itaewon office + Pangyo satellite office Additional Info: 전문연구요원 편입/전직 가능합니다. Hiring Process Application Review → Recruiter Interview (비대면/30분) → Loop Interview [Hiring Manager Interview&Live Coding Test Interview] (대면/약 90분) → System Design Interview(대면/약 90분) → Final Round Interview (비대면/약 45분) → Reference Check → Offer Benefits and Perks Growth & Tools 글로벌 B2B 고객과 함께 성장하는 Global Team 자율성과 협업을 모두 갖춘 하이브리드 근무 최신 맥북 및 70만 원 상당 재택근무 장비 지원, 3년 주기로 최신 장비 교체 Tokens never sleep - Tech 직군 LLM 토큰 무제한 지원 강의, 컨퍼런스, 멤버십 등에 사용 가능한 연 140만원 상당 자기개발비 지원 영어 교육 프로그램 및 글로벌 버디 프로그램 운영 야간 및 주말 출퇴근 택시비 지원 Meal & Snack 식비·교통비 등 자유롭게 사용할 수 있는 연 720만원 상당 법인카드 제공 사무실 내 스낵바 운영 (간식, 커피, 제철 과일 등) 사무실 근무 시, 오후 7시 이후 저녁 식대 제공 Wellness & Family 연 1회 본인 및 가족 1인의 건강검진 제공 단체보험 가입 (상해보험/치아보험/가족 상해보험 중 택 1) 독감 예방접종비 지원 연말 2주간 유급 Holiday Break 운영