AI Framework and Performance Experts(vLLM)

THE ROLE: As a senior member of the LLM inference framework team, you will be responsible for building and optimizing production-grade single-node and distributed inference runtimes for large language models on AMD GPUs. You will work at the framework and runtime layer, driving performance, scalability, and reliability, enabling tensor parallelism, pipeline parallelism, expert parallelism ( MoE ), and single-node or multi-node inference at scale. Your work will directly power customer-facing deployments and benchmarking platforms (e.g., InferenceMax , MLPerf , strategic partners, and cloud providers) and will be upstreamed into open-source inference frameworks such as vLLM and SGLang to make AMD a first-class platform for LLM serving. This role sits at the intersection of inference engines, distributed systems, and GPU runtime and kernel backends.

THE PERSON : You are a systems-minded ML engineer who thinks in terms of throughput, latency, memory movement, and scheduling, not just model code. You are comfortable reading and modifying large-scale inference frameworks, debugging performance across GPUs and nodes, and collaborating with kernel, compiler, and networking teams to close end-to-end performance gaps. You enjoy working in open source and driving architecture-level improvements in inference platforms.

KEY RESPONSIBILITIES: Inference Framework & Runtime

• Architect and optimize distributed LLM inference runtimes based on in-house LLM engines or open-source stacks such as  vLLM ,  SGLang , and  llm -d
• Design and improve TP / PP / EP ( MoE ) hybrid execution, including KV-cache management, attention dispatch, and token scheduling
• Implement and optimize multi-node inference pipelines using RCCL, RDMA, and collective-based execution

Performance & Scalability

• Drive throughput, latency, and memory efficiency across single-GPU and multi-GPU clusters
• Optimize continuous batching, speculative decoding, KV-cache paging, prefix caching, and multi-turn serving

…