Run gpt-oss-20b locally: memory, speed and cost
gpt-oss-20b has 21 billion parameters, 3.6 billion active per token (mixture of experts). At MXFP4 (gpt-oss native) it needs about 13 GB with 32K tokens of context, so the cheapest hardware here that runs it is the RTX 5060 Ti 16 GB (about 94 tokens a second). It scores 138 on Epoch AI’s Capabilities Index. Through an API it costs $0.018 / $0.09 per 1M tokens.
Memory needed
Weights plus context cache
| Quantisation | 8K context | 32K context | 128K context |
|---|---|---|---|
| Q4_K_M | 14 GB | 15 GB | 17 GB |
| Q5_K_M | 16 GB | 17 GB | 19 GB |
| Q8_0 | 24 GB | 25 GB | 27 GB |
| BF16 | 44 GB | 45 GB | 47 GB |
Estimates: weights at the stated bits per weight, plus the context (KV) cache, plus about 1 GB of overhead. Lower-bit quantisations save memory at some cost in quality.
gpt-oss-20b on each device
MXFP4 (gpt-oss native), 32K context
| Hardware | Memory | Fits? | Speed (estimate) |
|---|---|---|---|
| RTX 5090 | 32 GB | Fits | 254 tok/s |
| RTX 5080 | 16 GB | Fits | 170 tok/s |
| RTX 5070 Ti | 16 GB | Fits | 162 tok/s |
| RTX 5060 Ti 16 GB | 16 GB | Fits | 94 tok/s |
| RTX 4090 (used) | 24 GB | Fits | 176 tok/s |
| RTX 3090 (used) | 24 GB | Fits | 167 tok/s |
| RTX 3060 12 GB | 12 GB | Partly, with system RAM | 49 tok/s |
| RTX PRO 6000 Blackwell | 96 GB | Fits | 254 tok/s |
| Radeon RX 9070 XT | 16 GB | Fits | 126 tok/s |
| Radeon RX 7900 XTX | 24 GB | Fits | 170 tok/s |
| Radeon AI PRO R9700 | 32 GB | Fits | 136 tok/s |
| Intel Arc Pro B70 | 32 GB | Fits | 91 tok/s |
| Mac Studio M5 Max 128 GB | 128 GB | Fits | 119 tok/s |
| Mac Studio M5 Ultra 512 GB | 512 GB | Fits | 142 tok/s |
| Ryzen AI Max+ 395 128 GB | 128 GB | Fits | 54 tok/s |
| NVIDIA DGX Spark | 128 GB | Fits | 57 tok/s |
Generation speed estimated from memory bandwidth, calibrated against published llama.cpp results. Fastest here: RTX 5090, about 254 tokens a second. Try other settings in the calculator.
How capable is it?
Capability 138 (range 133–140) on the Epoch Capabilities Index. API models with a similar score: Gemini 3.5 Flash-Lite (145, $0.30 / $2.50) and MiniMax M3 (147, $0.30 / $1.20). Similar local models: Qwen3.5 9B (139), gpt-oss-120b (140), Gemma 4 26B-A4B (142).
Source: Epoch Capabilities Index by Epoch AI, CC BY 4.0, read 4 Nov 2026.
Or use it through an API
$0.018 / $0.09 per 1M input / output tokens through OpenRouter (gpt-oss-20b hosted prices). A customer chat assistant of 100M input and 20M output tokens a month costs about $3.60, so local hardware pays for itself only with heavy use. Licence: Apache 2.0.
Questions
How much VRAM does gpt-oss-20b need?
About 13 GB at MXFP4 (gpt-oss native) with 8K tokens of context, 13 GB with 32K and 16 GB with 128K. At 8-bit (Q8_0) it needs about 24 GB with 8K context.
Can gpt-oss-20b run on a Mac?
Yes. A Mac Studio M5 Max with 128GB runs it with 32K context at an estimated 119 tokens a second.
How fast is gpt-oss-20b on an RTX 4090?
About 176 tokens a second with 32K context, an estimate from the card’s memory bandwidth.
How capable is gpt-oss-20b?
It scores 138 on Epoch AI’s Capabilities Index (range 133–140), close to Gemini 3.5 Flash-Lite (145) and MiniMax M3 (147) among API models.
What does gpt-oss-20b cost through an API?
$0.018 / $0.09 per 1M input / output tokens through OpenRouter, the cheapest host we track. A customer chat assistant of 100M input and 20M output tokens a month would cost about $3.60.