On-prem models / Zhipu AI (Z.ai)
Run GLM-5.3 on-prem
Z.ai's flagship text model, a post-training upgrade of the GLM-5.2 base with large gains in coding, long-horizon agents and cyber-security tasks. Weights ship in FP8 with a separate BF16 repo.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 1488 GB | 0.74 GB | 2.94 GB | 11.8 GB |
| FP8 | 744 GB | 0.37 GB | 1.47 GB | 5.89 GB |
| INT4 (AWQ / GPTQ) | 417 GB | 0.37 GB | 1.47 GB | 5.89 GB |
| GGUF Q4_K_M | 446 GB | 0.37 GB | 1.47 GB | 5.89 GB |
KV cache from the published config: 78 layers, MLA latent 512 + 64 RoPE dims. 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Excellent for central on-prem agents that write SQL, reconcile data and drive ERP APIs, with a very permissive licence. Hardware cost puts it at group-IT scale; GLM-5.3-Flash is the more practical choice for most sites.
Strengths
- Among the strongest open coding and agent models at release
- 1M context with sparse attention
- Licence is effectively unrestricted for almost everyone
- Ascend NPU deployment documented as well as GPU
Watch out for
- About 750 GB in FP8; needs an 8x H200/B200-class node
- Text only
- Thinking defaults to max effort; set reasoning_effort and clear_thinking for chat use
- Strong offensive-security capability may matter for internal governance
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 8 | 430 / 460 GB | 90 |
| AMD Instinct MI325X 256GB | 2 | 421 / 460 GB | 130 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 4 | 424 / 460 GB | 116 |
| Apple Mac Studio M3 Ultra 512GB | 1 | 419 / 460 GB | 16 |
| Apple Mac Studio M5 Ultra 512GB | 1 | 419 / 460 GB | 23 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 2 | 421 / 486 GB | 153 |
| NVIDIA H200 SXM 141GB | 4 | 424 / 507 GB | 130 |
| NVIDIA H200 NVL 141GB | 4 | 424 / 507 GB | 130 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI325X 256GB | 4 | 765 / 921 GB | 102 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 8 | 771 / 921 GB | 94 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 4 | 765 / 972 GB | 118 |
| NVIDIA H200 SXM 141GB | 8 | 771 / 1015 GB | 104 |
| NVIDIA H200 NVL 141GB | 8 | 771 / 1015 GB | 104 |
| AMD Instinct MI355X 288GB | 4 | 765 / 1036 GB | 118 |
| NVIDIA B200 180GB | 8 | 771 / 1296 GB | 121 |
| AMD Instinct MI300X 192GB | 8 | 771 / 1382 GB | 108 |
Serving and licence
SGLang cookbook, vLLM recipe, TokenSpeed, Transformers (glm_moe_dsa) and KTransformers; vLLM-Ascend, xLLM and SGLang on Ascend NPUs.
GLM-5.3 License. Permissive MIT-style grant; only Model-as-a-Service operators with over US$10B annual revenue must pass a Z.ai security review before commercial use.
Design a GLM-5.3 deployment
Preloaded with GLM-5.3. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does GLM-5.3 need?
The weights alone take about 1488 GB in BF16, 744 GB in FP8 and 417 GB in 4-bit. On top of that each request holds a KV cache of about 0.37 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 7.36 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs GLM-5.3?
For a single user at 8K context in 4-bit, 8 x AMD Instinct MI210 64GB (64 GB each) fits, at an estimated 90 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is GLM-5.3 allowed for commercial use?
GLM-5.3 is released under GLM-5.3 License. Permissive MIT-style grant; only Model-as-a-Service operators with over US$10B annual revenue must pass a Z.ai security review before commercial use. Check the licence text on the model card with your legal team before production use.
Which engine should serve GLM-5.3 on-prem?
SGLang cookbook, vLLM recipe, TokenSpeed, Transformers (glm_moe_dsa) and KTransformers; vLLM-Ascend, xLLM and SGLang on Ascend NPUs.