Run Llama 3.3 70B locally: memory, speed and cost
Llama 3.3 70B has 70.6 billion parameters (dense). At Q4_K_M it needs about 55 GB with 32K tokens of context, so the cheapest hardware here that runs it is the Ryzen AI Max+ 395 128 GB (about 3.8 tokens a second). It scores 127 on Epoch AI’s Capabilities Index. Through an API it costs $0.10 / $0.32 per 1M tokens.
Memory needed
Weights plus context cache
| Quantisation | 8K context | 32K context | 128K context |
|---|---|---|---|
| Q4_K_Musual choice | 47 GB | 55 GB | 88 GB |
| Q5_K_M | 55 GB | 63 GB | 95 GB |
| Q8_0 | 80 GB | 88 GB | 120 GB |
| BF16 | 148 GB | 156 GB | 188 GB |
Estimates: weights at the stated bits per weight, plus the context (KV) cache, plus about 1 GB of overhead. Lower-bit quantisations save memory at some cost in quality.
Llama 3.3 70B on each device
Q4_K_M, 32K context
| Hardware | Memory | Fits? | Speed (estimate) |
|---|---|---|---|
| RTX 5090 | 32 GB | Partly, with system RAM | 2.3 tok/s |
| RTX 5080 | 16 GB | Partly, with system RAM | 1.4 tok/s |
| RTX 5070 Ti | 16 GB | Partly, with system RAM | 1.4 tok/s |
| RTX 5060 Ti 16 GB | 16 GB | Partly, with system RAM | 1.4 tok/s |
| RTX 4090 (used) | 24 GB | Partly, with system RAM | 1.8 tok/s |
| RTX 3090 (used) | 24 GB | Partly, with system RAM | 1.8 tok/s |
| RTX 3060 12 GB | 12 GB | Partly, with system RAM | 1.3 tok/s |
| RTX PRO 6000 Blackwell | 96 GB | Fits | 28 tok/s |
| Radeon RX 9070 XT | 16 GB | Partly, with system RAM | 1.4 tok/s |
| Radeon RX 7900 XTX | 24 GB | Partly, with system RAM | 1.8 tok/s |
| Radeon AI PRO R9700 | 32 GB | Partly, with system RAM | 2.2 tok/s |
| Intel Arc Pro B70 | 32 GB | Partly, with system RAM | 2.1 tok/s |
| Mac Studio M5 Max 128 GB | 128 GB | Fits | 9.7 tok/s |
| Mac Studio M5 Ultra 512 GB | 512 GB | Fits | 12.2 tok/s |
| Ryzen AI Max+ 395 128 GB | 128 GB | Fits | 3.8 tok/s |
| NVIDIA DGX Spark | 128 GB | Fits | 4.1 tok/s |
Generation speed estimated from memory bandwidth, calibrated against published llama.cpp results. Fastest here: RTX PRO 6000 Blackwell, about 28 tokens a second. Try other settings in the calculator.
How capable is it?
Capability 127 (range 123–129) on the Epoch Capabilities Index. API models with a similar score: Gemini 3.5 Flash-Lite (145, $0.30 / $2.50) and MiniMax M3 (147, $0.30 / $1.20). Similar local models: gpt-oss-20b (138), Qwen3.5 9B (139), gpt-oss-120b (140).
Source: Epoch Capabilities Index by Epoch AI, CC BY 4.0, read 4 Nov 2026.
Or use it through an API
$0.10 / $0.32 per 1M input / output tokens through OpenRouter (Llama 3.3 70B hosted prices). A customer chat assistant of 100M input and 20M output tokens a month costs about $16.40, so local hardware pays for itself only with heavy use.
Questions
How much VRAM does Llama 3.3 70B need?
About 47 GB at Q4_K_M with 8K tokens of context, 55 GB with 32K and 88 GB with 128K. At 8-bit (Q8_0) it needs about 80 GB with 8K context.
Can Llama 3.3 70B run on a Mac?
Yes. A Mac Studio M5 Max with 128GB runs it with 32K context at an estimated 9.7 tokens a second.
How fast is Llama 3.3 70B on an RTX 4090?
It doesn’t fit in the 4090’s 24GB with 32K context, so part of it runs from system RAM at about 1.8 tokens a second.
How capable is Llama 3.3 70B?
It scores 127 on Epoch AI’s Capabilities Index (range 123–129), close to Gemini 3.5 Flash-Lite (145) and MiniMax M3 (147) among API models.
What does Llama 3.3 70B cost through an API?
$0.10 / $0.32 per 1M input / output tokens through OpenRouter, the cheapest host we track. A customer chat assistant of 100M input and 20M output tokens a month would cost about $16.40.