On-prem models / Zhipu AI (Z.ai)
Run GLM-4.7-Flash on-prem
Lightweight 30B-A3B MoE from the GLM-4.7 generation using MLA attention. Z.ai pitches it as the strongest model in the 30B class for local deployment.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 62.4 GB | 0.44 GB | 1.77 GB | 7.10 GB |
| FP8 | 31.2 GB | 0.22 GB | 0.89 GB | 3.55 GB |
| INT4 (AWQ / GPTQ) | 17.5 GB | 0.22 GB | 0.89 GB | 3.55 GB |
| GGUF Q4_K_M | 18.7 GB | 0.22 GB | 0.89 GB | 3.55 GB |
KV cache from the published config: 47 layers, MLA latent 512 + 64 RoPE dims. 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
A good low-cost local agent for ERP tool calling and moderate text-to-SQL, with a clean licence. Needs a separate vision model for scanned documents.
Strengths
- Fits one 48 GB GPU at 4-bit or one 80 GB GPU in BF16/FP8
- Compact MLA KV cache
- Preserved thinking for multi-turn agents
- MIT licence
Watch out for
- Text only
- Engine support initially required main-branch vLLM/SGLang
- Smaller knowledge base than 100B+ models
GPUs that run it
One user, 8K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA A100 40GB PCIe | 1 | 33 / 36 GB | 168 |
| NVIDIA L4 24GB | 2 | 35 / 43 GB | 65 |
| NVIDIA A10 24GB | 2 | 35 / 43 GB | 113 |
| NVIDIA L40S 48GB | 1 | 33 / 43 GB | 105 |
| NVIDIA RTX A6000 48GB | 1 | 33 / 43 GB | 95 |
| NVIDIA RTX 6000 Ada 48GB | 1 | 33 / 43 GB | 115 |
| NVIDIA GeForce RTX 4090 24GB | 2 | 35 / 43 GB | 160 |
| NVIDIA GeForce RTX 5090 32GB | 2 | 35 / 57 GB | 218 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA L4 24GB | 2 | 44 / 43 GB | 30 |
| NVIDIA A10 24GB | 2 | 44 / 43 GB | 56 |
| NVIDIA L40S 48GB | 1 | 42 / 43 GB | 48 |
| NVIDIA RTX A6000 48GB | 1 | 42 / 43 GB | 43 |
| NVIDIA RTX 6000 Ada 48GB | 1 | 42 / 43 GB | 53 |
| NVIDIA GeForce RTX 4090 24GB | 2 | 44 / 43 GB | 85 |
| NVIDIA GeForce RTX 5090 32GB | 2 | 44 / 57 GB | 131 |
| AMD Instinct MI210 64GB | 1 | 42 / 57 GB | 83 |
Serving and licence
vLLM and SGLang (recent releases) with --tool-call-parser glm47; on Blackwell with SGLang use the triton attention backend per the card. Community GGUFs for llama.cpp/Ollama.
MIT. MIT licence: commercial use allowed without revenue or user caps.
Design a GLM-4.7-Flash deployment
Preloaded with GLM-4.7-Flash. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does GLM-4.7-Flash need?
The weights alone take about 62.4 GB in BF16, 31.2 GB in FP8 and 17.5 GB in 4-bit. On top of that each request holds a KV cache of about 0.22 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 4.44 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs GLM-4.7-Flash?
For a single user at 8K context in FP8, 1 x NVIDIA A100 40GB PCIe (40 GB each) fits, at an estimated 168 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is GLM-4.7-Flash allowed for commercial use?
GLM-4.7-Flash is released under MIT. MIT licence: commercial use allowed without revenue or user caps. Check the licence text on the model card with your legal team before production use.
Which engine should serve GLM-4.7-Flash on-prem?
vLLM and SGLang (recent releases) with --tool-call-parser glm47; on Blackwell with SGLang use the triton attention backend per the card. Community GGUFs for llama.cpp/Ollama.