Technical Program Manager, Inference

Anthropic’s Inference organization serves Claude to millions of users and enterprise customers, across our own products and the world’s largest cloud platforms, with the speed, reliability, and efficiency that frontier AI demands. The stack that makes this possible is deep and tightly coupled: accelerator kernels, an in-house inference engine that manages the entire token path for both production and research workloads, distributed routing, autoscaling, and capacity management. Every layer affects the others, often in ways that are hard to see in isolation. We build across GPUs, TPUs, and Trainium, and the complexity of building, testing, and shipping inference software grows with every platform we add.

We're looking for a Technical Program Manager to drive the programs that scale it. The work spans four closely coupled areas:

• Scaling and infrastructure. Routing, autoscaling, and capacity management for the serving fleet, and bringing new capacity online.

• Velocity. The toolchains, development environments, and CI/CD that let inference engineers build, test, and ship across every accelerator platform.

• Inference engine. The software between the accelerator kernels and the routing layer that serves every request.

• Performance. Improving throughput, latency, cost, and reliability across the stack, with correctness treated as part of performance.

…