Run Qwen3.5 397B-A17B locally: memory, speed and cost
Qwen3.5 397B-A17B has 397 billion parameters, 17 billion active per token (mixture of experts). At Q4_K_M it needs about 248 GB with 32K tokens of context, so the cheapest hardware here that runs it is the Mac Studio M5 Ultra 512 GB (about 47 tokens a second). It scores 147 on Epoch AI’s Capabilities Index. Through an API it costs $0.55 / $3.50 per 1M tokens.
Memory needed
Weights plus context cache
| Quantisation | 8K context | 32K context | 128K context |
|---|---|---|---|
| Q4_K_Musual choice | 247 GB | 248 GB | 251 GB |
| Q5_K_M | 289 GB | 290 GB | 293 GB |
| Q8_0 | 432 GB | 432 GB | 436 GB |
| BF16 | 811 GB | 812 GB | 815 GB |
Estimates: weights at the stated bits per weight, plus the context (KV) cache, plus about 1 GB of overhead. Lower-bit quantisations save memory at some cost in quality.
Qwen3.5 397B-A17B on each device
Q4_K_M, 32K context
| Hardware | Memory | Fits? | Speed (estimate) |
|---|---|---|---|
| RTX 5090 | 32 GB | Won’t fit | — |
| RTX 5080 | 16 GB | Won’t fit | — |
| RTX 5070 Ti | 16 GB | Won’t fit | — |
| RTX 5060 Ti 16 GB | 16 GB | Won’t fit | — |
| RTX 4090 (used) | 24 GB | Won’t fit | — |
| RTX 3090 (used) | 24 GB | Won’t fit | — |
| RTX 3060 12 GB | 12 GB | Won’t fit | — |
| RTX PRO 6000 Blackwell | 96 GB | Won’t fit | — |
| Radeon RX 9070 XT | 16 GB | Won’t fit | — |
| Radeon RX 7900 XTX | 24 GB | Won’t fit | — |
| Radeon AI PRO R9700 | 32 GB | Won’t fit | — |
| Intel Arc Pro B70 | 32 GB | Won’t fit | — |
| Mac Studio M5 Max 128 GB | 128 GB | Won’t fit | — |
| Mac Studio M5 Ultra 512 GB | 512 GB | Fits | 47 tok/s |
| Ryzen AI Max+ 395 128 GB | 128 GB | Won’t fit | — |
| NVIDIA DGX Spark | 128 GB | Won’t fit | — |
Generation speed estimated from memory bandwidth, calibrated against published llama.cpp results. Fastest here: Mac Studio M5 Ultra 512 GB, about 47 tokens a second. Try other settings in the calculator.
How capable is it?
Capability 147 (range 145–149) on the Epoch Capabilities Index. API models with a similar score: MiniMax M3 (147, $0.30 / $1.20) and Qwen3.7 Plus (147, $0.40 / $1.60). Similar local models: Qwen3.6 35B-A3B (144), Gemma 4 31B (143), Gemma 4 26B-A4B (142).
Source: Epoch Capabilities Index by Epoch AI, CC BY 4.0, read 4 Nov 2026.
Or use it through an API
$0.55 / $3.50 per 1M input / output tokens. A customer chat assistant of 100M input and 20M output tokens a month costs about $125, so local hardware pays for itself only with heavy use.
Questions
How much VRAM does Qwen3.5 397B-A17B need?
About 247 GB at Q4_K_M with 8K tokens of context, 248 GB with 32K and 251 GB with 128K. At 8-bit (Q8_0) it needs about 432 GB with 8K context.
Can Qwen3.5 397B-A17B run on a Mac?
Not fully on a 128GB Mac Studio M5 Max with 32K context; the 512GB M5 Ultra runs it at about 47 tokens a second.
How fast is Qwen3.5 397B-A17B on an RTX 4090?
It doesn’t fit on a 24GB RTX 4090 with 32K context.
How capable is Qwen3.5 397B-A17B?
It scores 147 on Epoch AI’s Capabilities Index (range 145–149), close to MiniMax M3 (147) and Qwen3.7 Plus (147) among API models.
What does Qwen3.5 397B-A17B cost through an API?
$0.55 / $3.50 per 1M input / output tokens. A customer chat assistant of 100M input and 20M output tokens a month would cost about $125.