NVIDIA Nemotron 3.5 Bolt and Super VL, explained: a 74B agent model in 22 GB, and what it needs to run

Nemotron is NVIDIA's family of open models. "Open" here means you can download the weights (the numbers the model learned) and, for Nemotron, much of the training data and the training recipes too. Before these came Nemotron 3.5 Lightning (30B total, 3B active), the first 3.5 chat model, whose files went up on Hugging Face in early August. On 15 October NVIDIA added two bigger ones, both vision-language models: they read text, images and video, and they write text.

Nemotron 3.5 Super VL has 120 billion parameters, of which 12 billion are active for each token. It is a mixture-of-experts model (MoE): instead of one big block of maths that every token passes through, each expert layer holds many smaller "experts" and a router picks a few for each token. So the model is large, but each token only costs the work of a 12B model. Super VL supports up to 1 million tokens of context, and at full precision NVIDIA lists it on 4 H100 GPUs or 2 GB200s. Its 4-bit version (NVFP4, NVIDIA's own 4-bit number format) runs on one H200, two H100s, one GB200 or one DGX Spark, NVIDIA's desktop AI computer.

Nemotron 3.5 Bolt 74B-A9B 24GB is the one aimed at your own machine. NVIDIA's card says it is "designed to run everyday agentic tasks locally, using a native 2-bit quantization to fit in 24GB of memory and run alongside other applications." Quantization means storing each weight in fewer bits than it was trained with; 2 bits is very aggressive. It has 74 billion parameters, 9 billion active per token, and up to 256K tokens of context.

Both are under OpenMDW-1.1, a short permissive licence from the Linux Foundation's OpenMDW project. It lets you use, change and share the model "without restriction", commercially too, as long as you keep the licence and notices when you redistribute it. NVIDIA's Hugging Face page also lists new Nemotron 3.5 Embed and Rerank models from the same weeks, which are used for search rather than chat.

NVIDIA's claim: for a fixed memory budget, it is better to shrink a large model than to train a small one from scratch. Bolt starts as Super VL (120B total, 12B active) and goes through five stages, all described on the model card.

First, compression with Star Elastic, an NVIDIA method that searches for a smaller shape automatically. A search step decides, layer by layer, how many experts to keep and how wide each one is, aiming at under 24 GB at a batch of one with 128K context. The number of experts each token uses drops from 22 to 9. The search and the retraining that follows it used 120 billion tokens in total, by the card's compression section. Second, the vision part from Super VL is attached again and only the small connector between vision and language is retrained. Third, supervised fine-tuning on examples, heavy on screens and web pages. Fourth, reinforcement learning on agent tasks: tool calls, MCP servers and software engineering. Fifth, the experts are stored in 2 bits. NVIDIA then trained the 2-bit model to copy the answers of the full-precision one (quantization-aware distillation), so it loses less accuracy.

The result has an unusual shape, which I could see in the file. The expert layers are not all alike: each keeps one of five expert counts, from 240 to 496, and one of four widths, from 1,536 to 2,560, chosen layer by layer by the search. And each expert works on a shrunk copy of the token: the token's 4,096 numbers are squeezed to 1,024 (NVIDIA calls this a latent space), the experts work on those, and the result is grown back, which makes each expert cheaper to store and run.

If quantization is new to you, my lesson "Quantization: The Same Model in Fewer Bits" in the AI Engineering course runs one model as a 16-bit, an 8-bit and a 4-bit file and measures what each costs in quality and saves in memory.

A token is a small piece of text, often part of a word. A normal Transformer, the design behind most language models, uses attention in every layer. Attention lets each new token look back at every earlier token, and to do that cheaply the model keeps a KV cache: a key and a value for every earlier token, in every attention layer. The KV cache grows with every token, so long conversations eat memory.

Bolt is a hybrid. Its file lists 88 blocks: 40 Mamba-2 blocks, 40 expert blocks and only 8 attention blocks. Mamba-2 is a state-space model: instead of keeping every earlier token, it keeps one running summary of fixed size and updates it as each token arrives. That summary is the same size at token 10 and at token 200,000. The expert blocks keep nothing per token at all.

The arithmetic from the header: each of the 8 attention blocks keeps 2 key-value heads of 128 numbers, so at 2 bytes a number (f16, 16-bit numbers) the whole model stores 8,192 bytes per token. A full 262,144-token conversation is 2.15 GB. The Mamba-2 state is 173 MB, whatever the length. If all 88 blocks were attention blocks like these, the same conversation would need 23.6 GB, 11 times more, which alone would be bigger than the model.

The price of this design is that a fixed-size summary cannot remember everything exactly, which is the usual reason hybrid designs keep a few attention blocks spread through the model. How well that works on your long documents is something only your own tests will tell you.

The model facts come from NVIDIA's Hugging Face model cards for Bolt and Super VL, NVIDIA's Bolt technical blog dated 15 October 2026, and Hugging Face's own file listings, all read on 17 October 2026 and linked below. The licence is the OpenMDW-1.1 text itself.

I did not send a prompt to Bolt or Super VL. Bolt's 24.75 GB package is over the 17.18 GB Metal recommends for my Mac's GPU, and downloading it would break the disk limit this machine keeps. So I cannot tell you how good its answers are or how fast it runs, only what NVIDIA reports: 84 tokens a second on a coding test on an RTX Spark laptop with its drafter.

What I did run is real: the GGUF header read by range request and every count from it, the memory arithmetic, Ollama's registry manifest, Metal's memory limit on my Mac, and NVIDIA's chat template in its three settings. I will update this page if I run the model.

That arithmetic is the skill worth keeping. After the AI Engineering course you can look at any new model, from its card and its files, and say before you download it whether it fits your machine, what the cache will cost at your context length, and roughly how fast it will write.