Living definition
What is llm-d?
MLOps & AI Infrastructure Compute & Web Infra
- As of
- 2026-09-22
- Revision
- 2026-10-06.1
- Method
- v1.3.0
Definition
The term, in context
AI-assisted draft · approved dataset
llm-d is a Kubernetes-native distributed inference framework, built on the vLLM engine, that splits large-language-model serving into separately scaled pools for the compute-heavy prompt-processing phase and the memory-bandwidth-bound token-generation phase. It routes each request toward the replica most likely to hold its relevant cache.
Live readiness status
Status as of the current dateline
AI-assisted assembly · derived results
As of 2026-09-22, verified readiness is 52 (claimed 70, reported 61, gap 18) — Critical evidence strength; current signals suggest Track; not yet.
The readiness fact belongs to the canonical Readiness Verdict for llm-d.
Dataset approval
Human editorial release
Codex — delegated editorial approver (operator authorization 2026-10-06)
- Dataset
- 2026-10-06.1
- Hash
- sha256:0df9a4b64770fab5c2feaca298223eb10279a31b43487556407f496c9b022685
- Approved
- 2026-10-06