Software Engineer, GenAI

Stack

Responsibilities

• ▹Design and build agentic systems that turn LLMs into composable, dependable tools (retrieval, tool use, structured outputs)
• ▹Collaborate with ML and infra engineers to scale and optimize agentic workflows (latency, context windows, model choice)
• ▹Write high-quality, modular code that's graceful under failure, flexible to change, and easy to iterate on
• ▹Own major architectural decisions: workflow structure, data flow, caching intermediate state, and structuring generative outputs
• ▹Drive rigorous evaluation: build benchmark datasets, develop automated and human-in-the-loop frameworks, design experiments for failure modes, run A/B tests
• ▹Leverage frontier capabilities: rapidly prototype with new models, open-source tools, and novel prompting techniques

Requirements

• ▹3+ years of experience building production-grade systems, including 1-2+ years focused on LLM-powered or agentic products
• ▹Deep fluency with LLM APIs, prompting strategies, and orchestration patterns (e.g., LangChain, LlamaIndex, custom pipelines)
• ▹Experience with retrieval systems (semantic and lexical retrieval, vector DBs, efficient kNN), function calling, tool-use, or agentic workflows
• ▹Working knowledge of model evaluation: building diverse datasets, automated and human-in-the-loop evaluations, A/B tests, collaborating with subject matter experts
• ▹Strong Python fundamentals: clean code, comprehensive test cases; async programming, performance profiling, packaging, and deployment tooling experience preferred
• ▹Willingness to work from the SF or NYC office at least 3x per week (hybrid model)

Nice to have

• ▹Good taste and intuition for when to move fast and iterate versus when to address tech debt

Soft skills

What we offer

• ▹14 paid holidays, flexible PTO for salaried employees
• ▹Comprehensive medical, dental, and vision coverage for full-time employees and their families
• ▹Generous HSA contribution on High Deductible Health Plans
• ▹Paid parental leave
• ▹Family-forming benefits
• ▹401(k) matching
• ▹Personal device allowance
• ▹Pre-tax benefits (FSA, commuter benefits)
• ▹Monthly lifestyle wallet (fitness, professional development, coworking)
• ▹Mental health support (therapy, coaching)
• ▹Paid sabbatical after 5 years
• ▹Competitive compensation and equity

About the company

Abridge was founded in 2018 with the mission of powering deeper understanding in healthcare. Its AI-powered platform transforms patient-clinician conversations into structured clinical notes in real time with deep EMR integration, and is the only company mapping AI-generated summaries back to ground-truth evidence.