On-prem models / DeepSeek
Run DeepSeek-V4-Pro-0813 on-prem
Official August 2026 release of DeepSeek-V4-Pro, a 1.6T-parameter MoE with 49B active and 1M context. Adds a DSpark speculative decoding module and stronger agentic behaviour over the April preview.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 3200 GB | 1.02 GB | 4.09 GB | 16.4 GB |
| FP8 | 1600 GB | 0.51 GB | 2.05 GB | 8.19 GB |
| INT4 (AWQ / GPTQ) | 896 GB | 0.51 GB | 2.05 GB | 8.19 GB |
| GGUF Q4_K_M | 960 GB | 0.51 GB | 2.05 GB | 8.19 GB |
KV cache from the published config: 61 layers, 1 KV heads x 512 dims, 0 sliding-window layers (128 tokens). 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Excellent quality for complex SQL generation, root-cause analysis over long logs and multi-step agents, and the MIT licence is clean. Hardware needs put it in data-centre territory, so most manufacturers will run V4.1-Flash or a mid-size model on-prem instead.
Strengths
- Among the strongest open models for coding agents and reasoning
- MIT licence with no usage caps
- Compressed attention makes 1M context practical
- Three reasoning effort levels (low, high, max)
Watch out for
- Weights are FP4 experts plus FP8 (roughly 0.9 TB); needs a GB300/B200-class multi-GPU node
- No Jinja chat template; prompts must be encoded with DeepSeek's Python encoder or deepseek-recipe
- Text only
- Max effort can emit very long outputs (card suggests up to 384K tokens)
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI325X 256GB | 4 | 903 / 921 GB | 144 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 8 | 909 / 921 GB | 129 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 4 | 903 / 972 GB | 161 |
| NVIDIA H200 SXM 141GB | 8 | 909 / 1015 GB | 140 |
| NVIDIA H200 NVL 141GB | 8 | 909 / 1015 GB | 140 |
| AMD Instinct MI355X 288GB | 4 | 903 / 1036 GB | 161 |
| NVIDIA B200 180GB | 8 | 909 / 1296 GB | 157 |
| AMD Instinct MI300X 192GB | 8 | 909 / 1382 GB | 143 |
20 people at once, 16K context, MXFP4
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI325X 256GB | 4 | 875 / 921 GB | 126 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 8 | 881 / 921 GB | 116 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 4 | 875 / 972 GB | 144 |
| NVIDIA H200 SXM 141GB | 8 | 881 / 1015 GB | 128 |
| NVIDIA H200 NVL 141GB | 8 | 881 / 1015 GB | 128 |
| AMD Instinct MI355X 288GB | 4 | 875 / 1036 GB | 144 |
| NVIDIA B200 180GB | 8 | 881 / 1296 GB | 148 |
| AMD Instinct MI300X 192GB | 8 | 881 / 1382 GB | 132 |
Serving and licence
vLLM recipe (example: single 4x GB300 node, --trust-remote-code --kv-cache-dtype fp8 --block-size 256, DSpark via --speculative-config method dspark); SGLang cookbook with flashinfer_mxfp4 MoE backend. No practical llama.cpp path.
MIT. MIT licence for code and weights: commercial use allowed without revenue or user caps.
Design a DeepSeek-V4-Pro-0813 deployment
Preloaded with DeepSeek-V4-Pro-0813. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does DeepSeek-V4-Pro-0813 need?
The weights alone take about 3200 GB in BF16, 1600 GB in FP8 and 896 GB in 4-bit. On top of that each request holds a KV cache of about 0.51 GB at 8K tokens of context (MXFP4 cache), so 20 people generating at once need roughly 10.2 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs DeepSeek-V4-Pro-0813?
For a single user at 8K context in 4-bit, 4 x AMD Instinct MI325X 256GB (256 GB each) fits, at an estimated 144 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is DeepSeek-V4-Pro-0813 allowed for commercial use?
DeepSeek-V4-Pro-0813 is released under MIT. MIT licence for code and weights: commercial use allowed without revenue or user caps. Check the licence text on the model card with your legal team before production use.
Which engine should serve DeepSeek-V4-Pro-0813 on-prem?
vLLM recipe (example: single 4x GB300 node, --trust-remote-code --kv-cache-dtype fp8 --block-size 256, DSpark via --speculative-config method dspark); SGLang cookbook with flashinfer_mxfp4 MoE backend. No practical llama.cpp path.