GPUs for on-prem AI / Apple
Apple Mac Studio M5 Ultra 512GB for LLMs
Current Mac Studio Ultra at 1.2 TB/s; 512GB needs the 36-core CPU / 80-core GPU chip, and Apple lists that option as coming late October 2026 (256GB orderable now). Apple publishes no TFLOPS; engine support limited to MLX, llama.cpp and similar.
desktop system | Thunderbolt 5 | Metal/MLX
Which models fit, and how many cards
Planning estimates. 4-bit column: one user at 8K context. FP8 column: 10 users at once at 16K context.
| Model | Size | 4-bit: cards | 4-bit: tok/s | FP8, 10 users: cards | FP8: tok/s per user |
|---|---|---|---|---|---|
| Muse Spark Open 816B-A42B | 816B / 42B | 1 | 22 | 2 (multi-node) | 19 |
| DeepSeek-V4.1-Flash-1017 | 748B / 16B | 1 | 55 | 2 (multi-node) | 47 |
| GLM-5.3 | 744B / 40B | 1 | 23 | 2 (multi-node) | 19 |
| Mistral Large 3 675B Instruct 2512 | 675B / 41B | 1 | 23 | 2 (multi-node) | 20 |
| DeepSeek-V3.2 | 671B / 37B | 1 | 25 | 2 (multi-node) | 21 |
| MiniMax-M3 | 428B / 23B | 1 | 39 | 1 | 19 |
| Qwen3.5-397B-A17B | 403B / 17B | 1 | 52 | 1 | 28 |
| Llama 4 Maverick 17B-128E Instruct | 400B / 17B | 1 | 50 | 1 | 21 |
| Trinity Large Thinking | 398B / 13B | 1 | 63 | 1 | 27 |
| GLM-5.3-Flash | 320B / 18B | 1 | 49 | 1 | 28 |
| MiMo-V2.6-Flash-RL | 309B / 15B | 1 | 58 | 1 | 32 |
| Hy3 | 295B / 21B | 1 | 41 | 1 | 15 |
| Command A+ (05-2026) | 218B / 25B | 1 | 36 | 1 | 18 |
| Step-3.7-Flash | 198B / 11B | 1 | 76 | 1 | 38 |
| Qwen3.8-Flash-Next | 180B / 6B | 1 | 128 | 1 | 69 |
| Mistral Medium 3.5 128B | 128B | 1 | 10 | 1 | 5 |
| Qwen3.5-122B-A10B | 125B / 10B | 1 | 84 | 1 | 46 |
| Ling-3.0-flash | 124B / 5.1B | 1 | 147 | 1 | 86 |
| NVIDIA Nemotron 3 Super 120B-A12B | 120B / 12B | 1 | 72 | 1 | 41 |
| Mistral Small 4 119B A6B (2603) | 119B / 6.5B | 1 | 120 | 1 | 65 |
| gpt-oss-120b | 117B / 5.1B | 1 | 144 | 1 | 72 |
| Command A (03-2025) | 111B | 1 | 11 | 1 | 6 |
| Llama 4 Scout 17B-16E Instruct | 109B / 17B | 1 | 50 | 1 | 21 |
| Qwen3-Coder-Next | 79.7B / 3B | 1 | 212 | 1 | 111 |
| Llama 3.3 70B Instruct | 70B | 1 | 18 | 1 | 8 |
| Qwen3.6-35B-A3B | 36B / 3B | 1 | 213 | 1 | 116 |
| Qwen3-32B | 32.8B | 1 | 36 | 1 | 16 |
| Olmo 3.1 32B Think | 32B | 1 | 37 | 1 | 18 |
| GLM-4.7-Flash | 31.2B / 3B | 1 | 207 | 1 | 89 |
| Gemma 4 31B IT | 30.7B | 1 | 38 | 1 | 17 |
| Qwen3-30B-A3B-Instruct-2507 | 30.5B / 3.3B | 1 | 188 | 1 | 65 |
| Granite 4.2 30B | 30B | 1 | 39 | 1 | 17 |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | 30B / 3B | 1 | 216 | 1 | 133 |
| Qwen3.8-27B | 27.8B | 1 | 43 | 1 | 23 |
| Gemma 3 27B IT | 27B | 1 | 43 | 1 | 21 |
| Gemma 4 26B A4B IT | 25.2B / 3.8B | 1 | 172 | 1 | 73 |
| Mistral Small 3.2 24B Instruct 2506 | 24B | 1 | 48 | 1 | 23 |
| Magistral Small 1.2 (2509) | 24B | 1 | 48 | 1 | 23 |
| Devstral Small 2 24B Instruct 2512 | 24B | 1 | 48 | 1 | 23 |
| gpt-oss-20b | 21B / 3.6B | 1 | 187 | 1 | 99 |
| Phi-4 | 14B | 1 | 77 | 1 | 30 |
| Gemma 4 12B IT | 11.95B | 1 | 89 | 1 | 41 |
| Qwen3.5-9B | 9.7B | 1 | 109 | 1 | 59 |
| Qwen3-VL-8B-Instruct | 8.8B | 1 | 114 | 1 | 45 |
| Granite 4.2 8B | 8B | 1 | 122 | 1 | 45 |
| Llama 3.1 8B Instruct | 8B | 1 | 123 | 1 | 49 |
| DeepSeek-V4-Pro-0813 | 1600B / 49B | 2 (multi-node) | 29 | 4 (multi-node) | 27 |
| MiMo-V2.6-Pro-RL | 1020B / 42B | 2 (multi-node) | 33 | 3 (multi-node) | 25 |
| Kimi-K2.6 | 1000B / 32B | 2 (multi-node) | 41 | 3 (multi-node) | 31 |
| Kimi-K2.7-Code | 1000B / 32B | 2 (multi-node) | 41 | 3 (multi-node) | 31 |
| Qwen3.8-2.4T-A95B | 2446B / 95B | 3 (multi-node) | 21 | 6 (multi-node) | 20 |
| Kimi-K3 | 2800B / 104B | 4 (multi-node) | 24 | 7 (multi-node) | 22 |
Design a deployment on Apple Mac Studio M5 Ultra 512GB
Preloaded with this hardware. Add your ERP, documents and users.
Questions
What is the largest model that fits on one Apple Mac Studio M5 Ultra 512GB?
Of the models we track, Muse Spark Open 816B-A42B (816B total, 42B active) fits on a single 512 GB card in 4-bit with one user at 8K context. More users or longer context need memory for the KV cache, so leave headroom.
How fast is Apple Mac Studio M5 Ultra 512GB for LLM inference?
Generation speed is mostly set by memory bandwidth: 1200 GB/s here. Each token reads the active weights once, so a model with 8 GB of active weights tops out near 150 tokens per second per stream before efficiency losses; batching many users raises total throughput.
Is Apple Mac Studio M5 Ultra 512GB a good choice for on-prem AI?
Current Mac Studio Ultra at 1.2 TB/s; 512GB needs the 36-core CPU / 80-core GPU chip, and Apple lists that option as coming late October 2026 (256GB orderable now). Apple publishes no TFLOPS; engine support limited to MLX, llama.cpp and similar.