Run Qwen3.5 9B locally: memory, speed and cost
Qwen3.5 9B has 9 billion parameters (dense). At Q4_K_M it needs about 7.6 GB with 32K tokens of context, so the cheapest hardware here that runs it is the RTX 3060 12 GB (about 45 tokens a second). It scores 139 on Epoch AI’s Capabilities Index. Through an API it costs $0.10 / $0.15 per 1M tokens.
Memory needed
Weights plus context cache
| Quantisation | 8K context | 32K context | 128K context |
|---|---|---|---|
| Q4_K_Musual choice | 6.8 GB | 7.6 GB | 11 GB |
| Q5_K_M | 7.8 GB | 8.6 GB | 12 GB |
| Q8_0 | 11 GB | 12 GB | 15 GB |
| BF16 | 20 GB | 20 GB | 24 GB |
Estimates: weights at the stated bits per weight, plus the context (KV) cache, plus about 1 GB of overhead. Lower-bit quantisations save memory at some cost in quality.
Qwen3.5 9B on each device
Q4_K_M, 32K context
| Hardware | Memory | Fits? | Speed (estimate) |
|---|---|---|---|
| RTX 5090 | 32 GB | Fits | 186 tok/s |
| RTX 5080 | 16 GB | Fits | 110 tok/s |
| RTX 5070 Ti | 16 GB | Fits | 103 tok/s |
| RTX 5060 Ti 16 GB | 16 GB | Fits | 55 tok/s |
| RTX 4090 (used) | 24 GB | Fits | 115 tok/s |
| RTX 3090 (used) | 24 GB | Fits | 107 tok/s |
| RTX 3060 12 GB | 12 GB | Fits | 45 tok/s |
| RTX PRO 6000 Blackwell | 96 GB | Fits | 186 tok/s |
| Radeon RX 9070 XT | 16 GB | Fits | 76 tok/s |
| Radeon RX 7900 XTX | 24 GB | Fits | 110 tok/s |
| Radeon AI PRO R9700 | 32 GB | Fits | 81 tok/s |
| Intel Arc Pro B70 | 32 GB | Fits | 55 tok/s |
| Mac Studio M5 Max 128 GB | 128 GB | Fits | 72 tok/s |
| Mac Studio M5 Ultra 512 GB | 512 GB | Fits | 89 tok/s |
| Ryzen AI Max+ 395 128 GB | 128 GB | Fits | 30 tok/s |
| NVIDIA DGX Spark | 128 GB | Fits | 32 tok/s |
Generation speed estimated from memory bandwidth, calibrated against published llama.cpp results. Fastest here: RTX 5090, about 186 tokens a second. Try other settings in the calculator.
How capable is it?
Capability 139 (range 136–141) on the Epoch Capabilities Index. API models with a similar score: Gemini 3.5 Flash-Lite (145, $0.30 / $2.50) and MiniMax M3 (147, $0.30 / $1.20). Similar local models: gpt-oss-20b (138), gpt-oss-120b (140), Gemma 4 26B-A4B (142).
Source: Epoch Capabilities Index by Epoch AI, CC BY 4.0, read 4 Nov 2026.
Or use it through an API
$0.10 / $0.15 per 1M input / output tokens. A customer chat assistant of 100M input and 20M output tokens a month costs about $13.00, so local hardware pays for itself only with heavy use.
Questions
How much VRAM does Qwen3.5 9B need?
About 6.8 GB at Q4_K_M with 8K tokens of context, 7.6 GB with 32K and 11 GB with 128K. At 8-bit (Q8_0) it needs about 11 GB with 8K context.
Can Qwen3.5 9B run on a Mac?
Yes. A Mac Studio M5 Max with 128GB runs it with 32K context at an estimated 72 tokens a second.
How fast is Qwen3.5 9B on an RTX 4090?
About 115 tokens a second with 32K context, an estimate from the card’s memory bandwidth.
How capable is Qwen3.5 9B?
It scores 139 on Epoch AI’s Capabilities Index (range 136–141), close to Gemini 3.5 Flash-Lite (145) and MiniMax M3 (147) among API models.
What does Qwen3.5 9B cost through an API?
$0.10 / $0.15 per 1M input / output tokens. A customer chat assistant of 100M input and 20M output tokens a month would cost about $13.00.