free tool · no sign-up · every formula shown
Cost per Million Tokens Calculator: your own GPU against an API
You pay for a GPU by the hour, busy or not. Divide that price by the tokens it writes in an hour and you have the price of a million tokens. The GPU prices are dated presets from RunPod, AWS and Google Cloud, and you can change any of them.
Model sizes, GPU specifications and prices were read from their official sources on 13 November 2026, and every number says where it came from.
What does a million tokens cost on your own GPU?
cost per million tokensYou pay for a GPU by the hour, whether it is busy or not. Divide that price by the tokens it writes in an hour and you get the price of a million tokens, which you can set against what an API charges.
Cost of one million tokens
$0.207
at 100% busy
Tokens written per second
1,059
estimated for 32 at once
GPUs you pay for
1
$0.790 an hour
Tokens in one paid hour
3.8 million
Cost per million tokens = price per hour x GPUs / (tokens a second x 3,600 x busy share) x 1,000,000
= $0.79 x 1 / (1,059 x 3,600 x 1.00) x 1,000,000 = $0.207
The assumptions behind these numbers
- The speed is the roofline estimate for a 1,024-token prompt and a 256-token answer at 85% of memory speed and 50% of compute speed. A measured number from a benchmark is always better; type it in above.
- It counts the tokens the model writes. Reading the prompt also takes GPU time, so a service with very long prompts and short answers costs more per written token than this shows.
- Prices are on-demand list prices for one GPU, read on 13 November 2026 from RunPod's public price list, AWS's us-east-1 price file and Google Cloud's us-central1 price page. Cloud machines include their CPUs and memory. Prices change often.
- Storage, network and the engineer who keeps it running are not counted.
The GPUs in this list: NVIDIA L4, RTX 4090, A10G (A10 specs), NVIDIA L40S, A100 40 GB, A100 80 GB SXM, H100 80 GB SXM, H200 141 GB SXM.
Three things to try
Drop the busy share to 30%
A GPU that waits for requests at night still costs the same per hour, so the cost of each token more than triples.
Serve more people at once
Moving from 8 to 64 people at once can cut the cost per token several times on the same GPU, as long as the KV cache fits.
Type in an API price
Enter what your provider charges for a million output tokens and see how fast your GPU must write to match it.
Questions people ask
How do I calculate the cost per million tokens?
Take the GPU's price per hour, multiply by the number of GPUs, and divide by the tokens they write in an hour (tokens a second x 3,600). Multiply by a million. One GPU at $2.69 an hour writing 1,000 tokens a second costs about $0.75 per million tokens.
Is it cheaper to host a model myself than to use an API?
Only if the GPU stays busy. A hosted API charges per token, so you pay nothing when nobody asks. Your own GPU costs the same every hour, so it wins when traffic is steady and high, and loses when it is quiet.
Where do the GPU prices come from?
RunPod's public price list, AWS's on-demand price file for us-east-1 and Google Cloud's price page for us-central1, all read on 13 November 2026. They are on-demand prices for one GPU; reserved and spot prices are lower, and prices change often.
Why does reading the prompt matter for cost?
This calculator counts the tokens a model writes. Reading a long prompt also uses the GPU, so a service with long prompts and short answers costs more per written token than the simple sum shows.
The other inference tools: GPU Memory Calculator for LLMs, LLM Latency and Throughput Estimator, KV Cache and PagedAttention Visualizer, Static and Continuous Batching Visualizer, Speculative Decoding Simulator. They come from the course Hands-On Inference Engineering, where each sum is worked on paper and then measured on a real GPU. To see a tool used on a real question, try the free exercises with worked answers.