Contents Part 9 · vLLM settings reference
Part 9 · vLLM settings reference
Memory and the KV cache in vLLM: every setting that decides what fits on the card
GPU memory share, maximum length, block size, KV cache data type, offloading and prefix caching, each with its default and a real use.
This lesson is part of the paid Inference Engineering course
The founding price is ₹2,222 or $99$49, one payment. It rises to ₹9,999 or $249 after the founding period, and buying now gives you every part as it is released, at no extra cost, with lifetime access.
Free to study first: Introduction: what this course builds, Before you start, Prefill and decode and The KV cache.
Already bought it? Sign in to read this lesson.