GPUs for on-prem AI / Apple
Apple Mac Studio M3 Ultra 512GB for LLMs
512GB unified memory runs 600B+ parameter models quantized on one box, but prompt processing is slow vs NVIDIA GPUs and serving relies on MLX or llama.cpp. Apple publishes no TFLOPS; 480W is max continuous system power. Apple has since dropped the 512GB option (256GB max now listed) and moved Mac Studio to M5 Ultra, so supply is limited.
desktop system | Thunderbolt 5 | Metal/MLX
Which models fit, and how many cards
Planning estimates. 4-bit column: one user at 8K context. FP8 column: 10 users at once at 16K context.
| Model | Size | 4-bit: cards | 4-bit: tok/s | FP8, 10 users: cards | FP8: tok/s per user |
|---|---|---|---|---|---|
| Muse Spark Open 816B-A42B | 816B / 42B | 1 | 15 | 2 (multi-node) | 13 |
| DeepSeek-V4.1-Flash-1017 | 748B / 16B | 1 | 39 | 2 (multi-node) | 34 |
| GLM-5.3 | 744B / 40B | 1 | 16 | 2 (multi-node) | 14 |
| Mistral Large 3 675B Instruct 2512 | 675B / 41B | 1 | 16 | 2 (multi-node) | 14 |
| DeepSeek-V3.2 | 671B / 37B | 1 | 17 | 2 (multi-node) | 15 |
| MiniMax-M3 | 428B / 23B | 1 | 27 | 1 | 13 |
| Qwen3.5-397B-A17B | 403B / 17B | 1 | 36 | 1 | 20 |
| Llama 4 Maverick 17B-128E Instruct | 400B / 17B | 1 | 35 | 1 | 14 |
| Trinity Large Thinking | 398B / 13B | 1 | 44 | 1 | 19 |
| GLM-5.3-Flash | 320B / 18B | 1 | 35 | 1 | 19 |
| MiMo-V2.6-Flash-RL | 309B / 15B | 1 | 41 | 1 | 22 |
| Hy3 | 295B / 21B | 1 | 28 | 1 | 10 |
| Command A+ (05-2026) | 218B / 25B | 1 | 25 | 1 | 13 |
| Step-3.7-Flash | 198B / 11B | 1 | 54 | 1 | 26 |
| Qwen3.8-Flash-Next | 180B / 6B | 1 | 93 | 1 | 49 |
| Mistral Medium 3.5 128B | 128B | 1 | 7 | 1 | 3 |
| Qwen3.5-122B-A10B | 125B / 10B | 1 | 59 | 1 | 32 |
| Ling-3.0-flash | 124B / 5.1B | 1 | 108 | 1 | 61 |
| NVIDIA Nemotron 3 Super 120B-A12B | 120B / 12B | 1 | 51 | 1 | 29 |
| Mistral Small 4 119B A6B (2603) | 119B / 6.5B | 1 | 87 | 1 | 46 |
| gpt-oss-120b | 117B / 5.1B | 1 | 106 | 1 | 51 |
| Command A (03-2025) | 111B | 1 | 8 | 1 | 4 |
| Llama 4 Scout 17B-16E Instruct | 109B / 17B | 1 | 35 | 1 | 14 |
| Qwen3-Coder-Next | 79.7B / 3B | 1 | 161 | 1 | 80 |
| Llama 3.3 70B Instruct | 70B | 1 | 12 | 1 | 6 |
| Qwen3.6-35B-A3B | 36B / 3B | 1 | 162 | 1 | 83 |
| Qwen3-32B | 32.8B | 1 | 25 | 1 | 11 |
| Olmo 3.1 32B Think | 32B | 1 | 26 | 1 | 12 |
| GLM-4.7-Flash | 31.2B / 3B | 1 | 157 | 1 | 63 |
| Gemma 4 31B IT | 30.7B | 1 | 26 | 1 | 12 |
| Qwen3-30B-A3B-Instruct-2507 | 30.5B / 3.3B | 1 | 141 | 1 | 46 |
| Granite 4.2 30B | 30B | 1 | 27 | 1 | 12 |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | 30B / 3B | 1 | 164 | 1 | 97 |
| Qwen3.8-27B | 27.8B | 1 | 30 | 1 | 16 |
| Gemma 3 27B IT | 27B | 1 | 30 | 1 | 15 |
| Gemma 4 26B A4B IT | 25.2B / 3.8B | 1 | 128 | 1 | 52 |
| Mistral Small 3.2 24B Instruct 2506 | 24B | 1 | 34 | 1 | 16 |
| Magistral Small 1.2 (2509) | 24B | 1 | 34 | 1 | 16 |
| Devstral Small 2 24B Instruct 2512 | 24B | 1 | 34 | 1 | 16 |
| gpt-oss-20b | 21B / 3.6B | 1 | 140 | 1 | 71 |
| Phi-4 | 14B | 1 | 54 | 1 | 21 |
| Gemma 4 12B IT | 11.95B | 1 | 63 | 1 | 29 |
| Qwen3.5-9B | 9.7B | 1 | 79 | 1 | 42 |
| Qwen3-VL-8B-Instruct | 8.8B | 1 | 82 | 1 | 31 |
| Granite 4.2 8B | 8B | 1 | 88 | 1 | 31 |
| Llama 3.1 8B Instruct | 8B | 1 | 89 | 1 | 34 |
| DeepSeek-V4-Pro-0813 | 1600B / 49B | 2 (multi-node) | 21 | 4 (multi-node) | 20 |
| MiMo-V2.6-Pro-RL | 1020B / 42B | 2 (multi-node) | 24 | 3 (multi-node) | 18 |
| Kimi-K2.6 | 1000B / 32B | 2 (multi-node) | 30 | 3 (multi-node) | 23 |
| Kimi-K2.7-Code | 1000B / 32B | 2 (multi-node) | 30 | 3 (multi-node) | 23 |
| Qwen3.8-2.4T-A95B | 2446B / 95B | 3 (multi-node) | 15 | 6 (multi-node) | 15 |
| Kimi-K3 | 2800B / 104B | 4 (multi-node) | 18 | 7 (multi-node) | 16 |
Design a deployment on Apple Mac Studio M3 Ultra 512GB
Preloaded with this hardware. Add your ERP, documents and users.
Questions
What is the largest model that fits on one Apple Mac Studio M3 Ultra 512GB?
Of the models we track, Muse Spark Open 816B-A42B (816B total, 42B active) fits on a single 512 GB card in 4-bit with one user at 8K context. More users or longer context need memory for the KV cache, so leave headroom.
How fast is Apple Mac Studio M3 Ultra 512GB for LLM inference?
Generation speed is mostly set by memory bandwidth: 819 GB/s here. Each token reads the active weights once, so a model with 8 GB of active weights tops out near 102 tokens per second per stream before efficiency losses; batching many users raises total throughput.
Is Apple Mac Studio M3 Ultra 512GB a good choice for on-prem AI?
512GB unified memory runs 600B+ parameter models quantized on one box, but prompt processing is slow vs NVIDIA GPUs and serving relies on MLX or llama.cpp. Apple publishes no TFLOPS; 480W is max continuous system power. Apple has since dropped the 512GB option (256GB max now listed) and moved Mac Studio to M5 Ultra, so supply is limited.