Senior Agent Evaluation Engineer, AI Evaluation Platform

Summary: Join Apple Services Engineering to define how we measure whether AI agents actually work. We are seeking a senior engineer who has built agents and knows how to evaluate them rigorously. You will design evaluation suites, build and validate judges, and ship agents on our evaluation platform, so the platform is shaped by real use and not guesswork. You will partner with researchers, product teams, and platform engineers to turn evaluation methodology into tools other teams can adopt. We are looking for builders who move quickly, iterate in the open, and can tell a trustworthy evaluation from a theatrical one.

Description: We're building the evaluation platform that will serve all of Apple's generative AI and agent systems. Building agents got faster. Knowing whether they got better did not. This role closes that gap.

You'll build real agents and evaluate them: multi-turn trajectories, tool-use chains, non-deterministic outputs, and judge reliability. You'll write evaluation suites, validate judges against human labels, and measure variance across trials. Some agents you build will evaluate other systems (test-case generation, judging, failure triage). Others will be reference agents or built with partner teams. When the platform gets in your way, you'll file the gap, and often fix it yourself.

This is a hands-on role with a lot of autonomy. You'll work in Python with a team of 7-12 engineers.

Key Responsibilities: Build and ship: Take ownership of features and services within the evaluation platform: APIs, SDKs, orchestration components, evaluation runners. You'll have the room to make calls on your own work and the support to deliver it well.

Productionize ML research: Partner with research engineers to take their prototype code and turn it into reliable services. You'll learn their world quickly and translate research patterns into clean Python that holds up under real load.

…