Senior Software Engineer, AI Agent & LLM

The Opportunity

We are looking for a Senior AI Agent & LLM Engineer who combines strong software engineering capabilities with a deep focus on AI quality. You will help build and improve the AI systems behind Otter’s conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform required to operate them reliably at scale.

This is a hands-on role for someone who can move from an ambiguous product problem to a working production solution, define how quality should be measured, and drive improvements across services, models, and user experience.

Your Impact

• Build AI-agent experiences for Otter AI Chat that help users reason over conversations, retrieve knowledge, and complete complex tasks.
• Advance Otter’s conversational knowledge engine through better knowledge extraction, annotation, and indexing.
• Develop robust evaluation datasets, automated graders, regression tests, and release gates for AI quality.
• Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and product workflows.
• Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability.
• Improve task completion, correctness, groundedness, reliability, latency, and cost.
• Turn production traces, customer feedback, and usage signals into measurable product and model improvements.
• Collaborate across AI, product, infrastructure, data, security, and application teams to deliver end-to-end capabilities.

We're looking for someone who

…