Contents Part 3 · vLLM in depth
Part 3 · vLLM in depth
Running vLLM as a real service: HTTPS, keys, a config file, request ids, logs and a graceful stop
Turning a test server into one other teams can safely call, with every service setting run and checked, including what happens to running answers when the server is stopped.
Code zip with this lesson
This lesson is part of the paid Inference Engineering course
The founding price is ₹2,222 or $99$49, one payment. It rises to ₹9,999 or $249 after the founding period, and buying now gives you every part as it is released, at no extra cost, with lifetime access.
Free to study first: Introduction: what this course builds, Before you start, Prefill and decode and The KV cache.
Already bought it? Sign in to read this lesson.