Best local AI models for the NVIDIA DGX Spark
128 GB of unified memory and 273 GB/s of bandwidth, for about $4,699. The most capable model that fits with 32K context is Qwen3.8 27B (capability 152), at about 11.6 tokens a second. 13 of 14 models fit.
Every model on the NVIDIA DGX Spark
Usual quantisation
| Model | Capability | 8K | 32K | 128K | Speed at 32K |
|---|---|---|---|---|---|
| Qwen3.8 27BQ4_K_M, 20 GB at 32K | 152 | Fits | Fits | Fits | 11.6 tok/s |
| Qwen3.6 35B-A3BQ4_K_M, 23 GB at 32K | 144 | Fits | Fits | Fits | 64 tok/s |
| Gemma 4 31BQ4_K_M, 23 GB at 32K | 143 | Fits | Fits | Fits | 10.2 tok/s |
| Gemma 4 26B-A4BQ4_K_M, 18 GB at 32K | 142 | Fits | Fits | Fits | 49 tok/s |
| gpt-oss-120bMXFP4 (gpt-oss native), 66 GB at 32K | 140 | Fits | Fits | Fits | 43 tok/s |
| Qwen3.5 9BQ4_K_M, 7.6 GB at 32K | 139 | Fits | Fits | Fits | 32 tok/s |
| gpt-oss-20bMXFP4 (gpt-oss native), 13 GB at 32K | 138 | Fits | Fits | Fits | 57 tok/s |
| Llama 3.3 70BQ4_K_M, 55 GB at 32K | 127 | Fits | Fits | Fits | 4.1 tok/s |
| Nemotron 3.5 Lightning 30B-A3BQ4_K_M, 20 GB at 32K | — | Fits | Fits | Fits | 80 tok/s |
| Mistral Small 4 119B-A6BQ4_K_M, 77 GB at 32K | — | Fits | Fits | Fits | 34 tok/s |
| Qwen3.5 122B-A10BQ4_K_M, 77 GB at 32K | — | Fits | Fits | Fits | 28 tok/s |
| Nemotron 3 Super 120B-A12BQ4_K_M, 76 GB at 32K | — | Fits | Fits | Fits | 26 tok/s |
| Muse Glimmer 30BQ4_K_M, 23 GB at 32K | — | Fits | Fits | Fits | 10.1 tok/s |
| Qwen3.5 397B-A17BQ4_K_M, 248 GB at 32K | 147 | Won’t fit | Won’t fit | Won’t fit | — |
Estimates from memory size and bandwidth, calibrated against published llama.cpp results. Capability: Epoch Capabilities Index by Epoch AI, CC BY 4.0, read 4 Nov 2026.
Similar-priced hardware
| Hardware | Memory | Bandwidth | Price (about) | Best model at 32K |
|---|---|---|---|---|
| NVIDIA DGX Spark | 128 GB | 273 GB/s | $4,699 | Qwen3.8 27B (152) |
| RTX 5090 | 32 GB | 1792 GB/s | $5,500 | Qwen3.8 27B (152) |
| RTX 4090 (used) | 24 GB | 1008 GB/s | $2,250 | Qwen3.8 27B (152) |
| Mac Studio M5 Max 128 GB | 128 GB | 614 GB/s | $4,500 | Qwen3.8 27B (152) |
| Ryzen AI Max+ 395 128 GB | 128 GB | 256 GB/s | $3,300 | Qwen3.8 27B (152) |
Questions
What is the best local LLM for the NVIDIA DGX Spark?
The highest-scoring model that fits with 32K tokens of context is Qwen3.8 27B (capability 152), at an estimated 11.6 tokens a second. For speed, Nemotron 3.5 Lightning 30B-A3B runs at about 80 tokens a second.
How many models fit on the NVIDIA DGX Spark?
13 of the 14 models we track fit in its 128 GB with 32K context, and 13 with 8K. 13 fit with 128K tokens of context for long documents.
What does the NVIDIA DGX Spark cost to run?
At about 200 W under load and $0.25 per kWh, about $0.05 an hour of generation. Against the API price of Qwen3.8 27B, at 4M input and 1M output tokens a day it would pay for itself in about 3070 months.