SGLang vs vLLM: What Is Different, and How to Choose
What is the difference between SGLang and vLLM?
Both are open-source model servers with an OpenAI-compatible API, continuous batching, paged KV cache memory, prefix reuse, chunked prefill, speculative decoding, tensor parallelism and quantization. The main difference is inside: vLLM introduced PagedAttention and reuses prompt beginnings by hashing pages of 16 tokens, while SGLang introduced RadixAttention, a tree of shared prompt beginnings, plus its own frontend language. Which is faster depends on your model, GPU and workload, so benchmark both fairly on your own traffic.
Save your progress and get the AI Engineering cheat sheet PDF
This guide compares the two open-source engines most teams look at for serving open models on GPUs. It answers 17 questions about what they share, where they differ inside, how to benchmark them without fooling yourself, and how to choose. The measured numbers are from vLLM runs on a rented RTX 4090; we say plainly where we have not measured SGLang yet.
17 questions · 4 sections · answers from the course
These answers come from our course Hands-On Inference Engineering: Serve LLMs on GPUs with vLLM and SGLang: 43 lessons ready so far, each one run and measured on a rented NVIDIA GPU. The first lessons are free to read, with no card needed.


The short version
Treat them as two good implementations of the same ideas. If your traffic resends long shared beginnings, SGLang's prefix tree is worth a test. If you want the broadest support and the most documentation, vLLM is the safer default. Either way, decide with your own prompts, fresh in every run, at your real load, and compare the slow end of the latency as well as total tokens a second.
If this helped, the full hands-on course goes much deeper.
In the course you serve the same model with vLLM on a rented GPU, learn every setting that changes a number, and then run a fair head-to-head benchmark against SGLang yourself.
1. The short answer
vLLM and SGLang are the two open-source engines most teams consider for serving open models on their own NVIDIA GPUs. They are more alike than the arguments online suggest.
What are SGLang and vLLM?
Both are model servers: programs that load a language model onto one or more GPUs, keep it in memory, and answer many people's requests at the same time. Both are free and open source, both are written mainly in Python with fast GPU code underneath, and both answer in the same format as OpenAI's API.
vLLM came out of the University of California, Berkeley, together with the 2023 paper that introduced PagedAttention. SGLang came out of the LMSYS research group and introduced RadixAttention, a way of reusing the work for prompts that start the same way. The course teaches vLLM 0.33 and SGLang 0.5.23, the versions it checked on 12 November 2026.
Is SGLang faster than vLLM?
It depends on the model, the GPU and above all the workload, and benchmarks published by different teams point in different directions. Workloads where many requests share long beginnings, such as long multi-turn chats or agents that send the same instructions again and again, are where prefix reuse matters most. Short, unrelated prompts depend more on each engine's GPU kernels and scheduler.
We have not yet run SGLang in the course, so we make no speed claim here. The fair comparison is one of the things the course's SGLang part measures. If someone shows you a single chart, ask how it was measured, using the rules further down this page.
Can I switch between vLLM and SGLang without changing my code?
Mostly, yes. Both serve OpenAI-compatible addresses such as /v1/chat/completions, so the OpenAI Python library works with either once you change the address and the model's name. In our vLLM run, the library sent a question to Qwen3-0.6B on an RTX 4090 and got the first word back after 0.167 seconds, with the token counts in the usual usage field.
What does change is the server side: the command that starts it, the names of its settings, and some extra request fields each engine accepts beyond OpenAI's own. Test those before you switch a live service.
What we measured: vLLM 0.33, Qwen3-0.6B, RTX 4090: first word after 0.167 s through the OpenAI Python library.
2. Where they differ inside
The real differences are in how each engine keeps and reuses the KV cache, how its scheduler is tuned, and what it offers beyond a plain server.

How does RadixAttention differ from vLLM's prefix caching?
Both reuse the notes of a prompt beginning the server has already read, so it is not read again. vLLM gives each full page of 16 tokens a label made from the page before it and its own tokens, and reuses pages from the start of a new prompt until the first page that differs. SGLang keeps the cached beginnings in a radix tree, a tree in which requests that share a beginning share a branch, and looks up the longest matching branch.
The goal is the same. vLLM's scheme was enough to make our first word about 7 times faster on a hit: eight questions behind the same 2,958-token handbook found 87% of their prompt in the cache, and the first word came after about 46 ms instead of 326 ms.
What we measured: vLLM 0.33 prefix caching on an RTX 4090: 87% of prompt tokens found, first word about 46 ms against 326 ms.
What breaks prefix reuse in either engine?
Anything that changes near the start of the prompt: a timestamp, a request number, a user's name, or a shuffled list of tools. Both engines can only reuse a beginning that matches token for token.
In our vLLM run, a clock time at the start of the system message cut the hit rate from 87% to 0%, and the same line at the end gave 86%. Put the parts that never change first, whichever engine you choose.
What we measured: vLLM 0.33: hit rate 87% with the time at the end of the system message, 0% with it at the start.
Do both engines store the KV cache in pages?
Both avoid reserving room for the longest possible conversation, which would waste most of the memory. vLLM's PagedAttention uses pages of 16 tokens by default, listed for each conversation in a block table. SGLang keeps cache entries in a shared memory pool indexed by its radix tree.
On our RTX 4090, vLLM made 2,076 pages for Qwen3-8B, and 64 conversations of about 340 tokens used only 66.8% of them at once, where reserving 32,768 tokens each would have fitted just one.
What we measured: vLLM 0.33, Qwen3-8B, RTX 4090: 2,076 pages of 16 tokens; 64 conversations filled 66.8%.
How do vLLM's main settings map to SGLang's?
The ideas match even where the names do not. vLLM's --gpu-memory-utilization, the share of the GPU it may use (0.92 by default in vLLM 0.33), corresponds to SGLang's --mem-fraction-static. vLLM's --max-num-seqs, the most conversations running at once, corresponds to SGLang's --max-running-requests. vLLM's --max-num-batched-tokens, the token budget of one step, plays the part of SGLang's --chunked-prefill-size, and both engines have a scheduling policy setting.
Defaults differ between the engines and between versions, so read each engine's start-up log to see how much room it gave the KV cache before you compare anything.
Do both engines support JSON schemas and tool calling?
Yes. Both can force the model's answer to follow a JSON schema or a grammar at serving time, by allowing only the tokens that keep the answer valid. Both can also parse tool calls from the model's output into OpenAI's tool-call format.
SGLang also comes with its own frontend language, a way of writing programs that make several model calls with branches and forced formats, which is where its prefix tree pays off. vLLM keeps to the server and the OpenAI-style API, with many settings for parsers and backends.
Lesson: Structured output, tool calling and reasoning in vLLM
Do both support speculative decoding?
Yes. In speculative decoding a small, fast guesser proposes several tokens and the large model checks them all in one pass, which can make one person's answer faster. vLLM 0.33 accepts methods from simple n-gram matching to a draft model, EAGLE-3 heads and multi-token prediction, and SGLang supports EAGLE-style guessing too.
It helps most when few people are using the GPU. On a busy server the extra checking work can make things slower, so measure it at your real load.
If this helped, the full hands-on course goes much deeper.
In the course you serve the same model with vLLM on a rented GPU, learn every setting that changes a number, and then run a fair head-to-head benchmark against SGLang yourself.
3. How to compare vLLM and SGLang fairly
Most comparisons online are not wrong on purpose. They just change more than one thing at a time. These rules keep a comparison honest.
What are the rules for a fair vLLM vs SGLang benchmark?
Use the same model files, the same number format, the same GPU and the same driver for both. Give both the same share of GPU memory and check in each log how much room the KV cache really got. Send the same workload: the same prompt and answer lengths, the same number of people at once or the same arrival rate, and the same sampling settings.
Warm both servers up first, run each test several times, and report the spread as well as the best run. Compare the median and p99 wait for the first word, the median and p99 gap between words, and total tokens a second, all at the same load. One number on its own hides the trade.
Why must every benchmark run use fresh prompts?
Because both engines remember prompts they have read. A test that sends the same questions again measures the prefix cache, not the GPU: the questions cost almost nothing to read the second time, and our first-word wait fell from 326 ms to 46 ms on a hit.
Give every run its own made-up questions, unless the thing you are deliberately measuring is reuse, and in vLLM check that vllm:prefix_cache_hits_total stays at 0 during the test.
Why should you compare second starts, not first starts?
Because a first start does extra work and can change the capacity you measure. vLLM compiles the model and measures its working space during the first start. On our card, a first start of Qwen3-8B got 33,216 tokens of KV cache and the next start with the same settings got 38,000.
If one engine is measured on a cold start and the other on a warm one, you are partly comparing start-up work. Start each engine once, then measure on the next start.
What we measured: vLLM 0.33, Qwen3-8B, RTX 4090, same settings: 33,216 tokens of KV cache on the first start, 38,000 on the next.
How many requests does a benchmark need?
Enough for the slow end to mean something. The p99 is the value 99 in 100 requests beat, so with a few dozen requests it is simply the slowest one. In our vLLM test with 48 answers, that was exactly what the p99 was.
Use hundreds of requests per level and say how many each percentile came from. Also check that the failed request count is 0, and know what your benchmark tool sends: vLLM 0.33's vllm bench serve sends no temperature unless you add one.
4. Choosing one
If you have to pick today, these questions usually settle it faster than a benchmark chart.
When is vLLM the safer choice?
When you want the engine with the widest set of supported models, settings and integrations, and the most material to learn from. Many other systems build on it: llm-d, for example, runs vLLM underneath its routing layer on Kubernetes. Its settings are documented one by one, and the course's vLLM reference explains every setting of version 0.33.
It is also the engine the course uses for most of its measurements, so every number you read here can be reproduced with it.
When is SGLang worth trying?
When your traffic reuses long beginnings heavily, for example agents that resend the same long instructions and history on every turn, many branches from one prompt, or programs that make several structured calls in a row. That is the kind of work RadixAttention was designed for.
Try it on your own workload with the fair rules above. If it keeps your promises at a lower cost, the switch is mostly a server change, because your client code speaks the same OpenAI-style API.
Which one should I learn first?
Learn the ideas first, because they carry over: prefill and decode, the KV cache, batching, paging and prefix reuse. Then learn one engine deeply. vLLM is a good first engine because so much is built on it, and once you know why each of its settings exists, SGLang's settings read like the same ideas with different names.
That is the order the course follows: the maths of one request, then vLLM in depth, then SGLang and a fair head-to-head benchmark against vLLM on the same GPU.
Where to go next
If this helped, the full hands-on course goes much deeper.
In the course you serve the same model with vLLM on a rented GPU, learn every setting that changes a number, and then run a fair head-to-head benchmark against SGLang yourself.
Every number on this page comes from Hands-On Inference Engineering: Serve LLMs on GPUs with vLLM and SGLang, where you rent the GPU yourself, run each command and compare your numbers with ours. Start with the free introduction and the free lessons on prefill and decode and the KV cache. If you are also preparing for interviews, the two big courses together and the Inference Engineering interview pack are below.
These questions are a start. The interview asks more.
Get all 120 questions in the Inference Engineering pack
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (141 diagrams in all). A PDF you keep, junior to staff level.
Pay by UPI, cards or netbanking through Razorpay. Pay by card or PayPal, in US dollars. Download at once from My packs.
