GPUs for on-prem AI / NVIDIA
NVIDIA DGX Spark (GB10) 128GB for LLMs
Desktop GB10 box with 128GB unified memory for prototyping models up to about 200B parameters (400B with two units); 273 GB/s makes decode slow vs data center GPUs. NVIDIA publishes only 1 PFLOP FP4 with sparsity, no BF16/FP8 figure; 240W PSU, 140W GB10 TDP. Founders Edition MSRP 4,699 USD since Feb 2026. Price is an indicative 2026 street range for one unit, not a quote.
desktop system | ConnectX-7 200 Gb/s (cluster 2 units) | CUDA | indicative price $4,000-4,700
Which models fit, and how many cards
Planning estimates. 4-bit column: one user at 8K context. FP8 column: 10 users at once at 16K context.
| Model | Size | 4-bit: cards | 4-bit: tok/s | FP8, 10 users: cards | FP8: tok/s per user |
|---|---|---|---|---|---|
| Step-3.7-Flash | 198B / 11B | 1 | 19 | 2 (multi-node) | 15 |
| Qwen3.8-Flash-Next | 180B / 6B | 1 | 34 | 2 (multi-node) | 27 |
| Mistral Medium 3.5 128B | 128B | 1 | 2 | 2 (multi-node) | 2 |
| Qwen3.5-122B-A10B | 125B / 10B | 1 | 21 | 2 (multi-node) | 18 |
| Ling-3.0-flash | 124B / 5.1B | 1 | 40 | 2 (multi-node) | 34 |
| NVIDIA Nemotron 3 Super 120B-A12B | 120B / 12B | 1 | 18 | 2 (multi-node) | 16 |
| Mistral Small 4 119B A6B (2603) | 119B / 6.5B | 1 | 32 | 2 (multi-node) | 26 |
| gpt-oss-120b | 117B / 5.1B | 1 | 39 | 2 (multi-node) | 29 |
| Command A (03-2025) | 111B | 1 | 3 | 2 (multi-node) | 2 |
| Llama 4 Scout 17B-16E Instruct | 109B / 17B | 1 | 12 | 2 (multi-node) | 8 |
| Qwen3-Coder-Next | 79.7B / 3B | 1 | 64 | 1 | 29 |
| Llama 3.3 70B Instruct | 70B | 1 | 4 | 1 | 2 |
| Qwen3.6-35B-A3B | 36B / 3B | 1 | 64 | 1 | 30 |
| Qwen3-32B | 32.8B | 1 | 9 | 1 | 4 |
| Olmo 3.1 32B Think | 32B | 1 | 9 | 1 | 4 |
| GLM-4.7-Flash | 31.2B / 3B | 1 | 62 | 1 | 22 |
| Gemma 4 31B IT | 30.7B | 1 | 9 | 1 | 4 |
| Qwen3-30B-A3B-Instruct-2507 | 30.5B / 3.3B | 1 | 55 | 1 | 16 |
| Granite 4.2 30B | 30B | 1 | 9 | 1 | 4 |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | 30B / 3B | 1 | 65 | 1 | 36 |
| Qwen3.8-27B | 27.8B | 1 | 10 | 1 | 5 |
| Gemma 3 27B IT | 27B | 1 | 10 | 1 | 5 |
| Gemma 4 26B A4B IT | 25.2B / 3.8B | 1 | 49 | 1 | 18 |
| Mistral Small 3.2 24B Instruct 2506 | 24B | 1 | 12 | 1 | 5 |
| Magistral Small 1.2 (2509) | 24B | 1 | 12 | 1 | 5 |
| Devstral Small 2 24B Instruct 2512 | 24B | 1 | 12 | 1 | 5 |
| gpt-oss-20b | 21B / 3.6B | 1 | 54 | 1 | 25 |
| Phi-4 | 14B | 1 | 19 | 1 | 7 |
| Gemma 4 12B IT | 11.95B | 1 | 23 | 1 | 10 |
| Qwen3.5-9B | 9.7B | 1 | 28 | 1 | 14 |
| Qwen3-VL-8B-Instruct | 8.8B | 1 | 30 | 1 | 11 |
| Granite 4.2 8B | 8B | 1 | 32 | 1 | 11 |
| Llama 3.1 8B Instruct | 8B | 1 | 33 | 1 | 12 |
| Qwen3.5-397B-A17B | 403B / 17B | 2 (multi-node) | 20 | 4 (multi-node) | 20 |
| Llama 4 Maverick 17B-128E Instruct | 400B / 17B | 2 (multi-node) | 20 | 4 (multi-node) | 15 |
| Trinity Large Thinking | 398B / 13B | 2 (multi-node) | 24 | 4 (multi-node) | 19 |
| GLM-5.3-Flash | 320B / 18B | 2 (multi-node) | 20 | 3 (multi-node) | 15 |
| MiMo-V2.6-Flash-RL | 309B / 15B | 2 (multi-node) | 23 | 3 (multi-node) | 17 |
| Hy3 | 295B / 21B | 2 (multi-node) | 16 | 3 (multi-node) | 8 |
| Command A+ (05-2026) | 218B / 25B | 2 (multi-node) | 15 | 2 (multi-node) | 8 |
| MiniMax-M3 | 428B / 23B | 3 (multi-node) | 20 | 4 (multi-node) | 13 |
| DeepSeek-V4.1-Flash-1017 | 748B / 16B | 4 (multi-node) | 38 | 7 (multi-node) | 34 |
| GLM-5.3 | 744B / 40B | 4 (multi-node) | 16 | 7 (multi-node) | 14 |
| Mistral Large 3 675B Instruct 2512 | 675B / 41B | 4 (multi-node) | 16 | 6 (multi-node) | 12 |
| DeepSeek-V3.2 | 671B / 37B | 4 (multi-node) | 18 | 6 (multi-node) | 13 |
| Kimi-K2.6 | 1000B / 32B | 5 (multi-node) | 22 | 9 (multi-node) | 17 |
| Kimi-K2.7-Code | 1000B / 32B | 5 (multi-node) | 22 | 9 (multi-node) | 17 |
| Muse Spark Open 816B-A42B | 816B / 42B | 5 (multi-node) | 18 | 8 (multi-node) | 15 |
| MiMo-V2.6-Pro-RL | 1020B / 42B | 6 (multi-node) | 20 | 10 (multi-node) | 15 |
| DeepSeek-V4-Pro-0813 | 1600B / 49B | 8 (multi-node) | 23 | 15 (multi-node) | 18 |
| Qwen3.8-2.4T-A95B | 2446B / 95B | 13 (multi-node) | 15 | 22 (multi-node) | 14 |
| Kimi-K3 | 2800B / 104B | 14 (multi-node) | 15 | 25 (multi-node) | 15 |
Design a deployment on NVIDIA DGX Spark (GB10) 128GB
Preloaded with this hardware. Add your ERP, documents and users.
Questions
What is the largest model that fits on one NVIDIA DGX Spark (GB10) 128GB?
Of the models we track, Step-3.7-Flash (198B total, 11B active) fits on a single 128 GB card in 4-bit with one user at 8K context. More users or longer context need memory for the KV cache, so leave headroom.
How fast is NVIDIA DGX Spark (GB10) 128GB for LLM inference?
Generation speed is mostly set by memory bandwidth: 273 GB/s here. Each token reads the active weights once, so a model with 8 GB of active weights tops out near 34 tokens per second per stream before efficiency losses; batching many users raises total throughput.
Is NVIDIA DGX Spark (GB10) 128GB a good choice for on-prem AI?
Desktop GB10 box with 128GB unified memory for prototyping models up to about 200B parameters (400B with two units); 273 GB/s makes decode slow vs data center GPUs. NVIDIA publishes only 1 PFLOP FP4 with sparsity, no BF16/FP8 figure; 240W PSU, 140W GB10 TDP. Founders Edition MSRP 4,699 USD since Feb 2026. Price is an indicative 2026 street range for one unit, not a quote.