On-prem models / DeepSeek
Run DeepSeek-V4.1-Flash-1017 on-prem
October 2026 minor update of DeepSeek-V4.1-Flash, a multimodal MoE built around extreme KV-cache compression (CSA2, FP4 KV, causal encoder-decoder). It natively reads images and text and targets input-heavy agent workloads.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 1496 GB | 0.08 GB | 0.28 GB | 1.08 GB |
| FP8 | 748 GB | 0.04 GB | 0.14 GB | 0.54 GB |
| INT4 (AWQ / GPTQ) | 419 GB | 0.04 GB | 0.14 GB | 0.54 GB |
| GGUF Q4_K_M | 449 GB | 0.04 GB | 0.14 GB | 0.54 GB |
KV cache from the published config: 40 layers, 1 KV heads x 512 dims, 36 sliding-window layers (128 tokens). 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Well suited to agents that ingest large volumes of ERP context (long order histories, specifications, scanned documents) because prompt processing is cheap. The hardware floor is still a serious multi-GPU server, so it fits central IT rather than a plant closet.
Strengths
- Tiny global KV cache, so 1M context and many concurrent sessions are affordable
- Low active compute, especially for long prompts
- Native image input
- MIT licence
Watch out for
- Large stored weight set (about 0.76T elements, FP4 experts plus FP8); needs a multi-GPU node
- Brand-new architecture: requires latest vLLM/SGLang builds and DeepSeek's encoding scripts (no Jinja template)
- Engram tables and DSpark add operational complexity
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 8 | 431 / 460 GB | 171 |
| AMD Instinct MI325X 256GB | 2 | 422 / 460 GB | 241 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 4 | 425 / 460 GB | 213 |
| Apple Mac Studio M3 Ultra 512GB | 1 | 421 / 460 GB | 39 |
| Apple Mac Studio M5 Ultra 512GB | 1 | 421 / 460 GB | 55 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 2 | 422 / 486 GB | 271 |
| NVIDIA H200 SXM 141GB | 4 | 425 / 507 GB | 231 |
| NVIDIA H200 NVL 141GB | 4 | 425 / 507 GB | 231 |
20 people at once, 16K context, MXFP4
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 8 | 410 / 460 GB | 169 |
| AMD Instinct MI325X 256GB | 2 | 401 / 460 GB | 238 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 4 | 404 / 460 GB | 211 |
| Apple Mac Studio M3 Ultra 512GB | 1 | 400 / 460 GB | 26 |
| Apple Mac Studio M5 Ultra 512GB | 1 | 400 / 460 GB | 26 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 2 | 401 / 486 GB | 268 |
| NVIDIA H200 SXM 141GB | 4 | 404 / 507 GB | 229 |
| NVIDIA H200 NVL 141GB | 4 | 404 / 507 GB | 229 |
Serving and licence
Use DeepSeek's vLLM/SGLang recipes for V4.1 (latest releases); prompts via the encoding folder or deepseek-recipe. Set max_tokens >= 256K per the card for long reasoning. No llama.cpp support confirmed.
MIT. MIT licence for code and weights: commercial use allowed without revenue or user caps.
Design a DeepSeek-V4.1-Flash-1017 deployment
Preloaded with DeepSeek-V4.1-Flash-1017. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does DeepSeek-V4.1-Flash-1017 need?
The weights alone take about 1496 GB in BF16, 748 GB in FP8 and 419 GB in 4-bit. On top of that each request holds a KV cache of about 0.04 GB at 8K tokens of context (MXFP4 cache), so 20 people generating at once need roughly 0.77 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs DeepSeek-V4.1-Flash-1017?
For a single user at 8K context in 4-bit, 8 x AMD Instinct MI210 64GB (64 GB each) fits, at an estimated 171 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is DeepSeek-V4.1-Flash-1017 allowed for commercial use?
DeepSeek-V4.1-Flash-1017 is released under MIT. MIT licence for code and weights: commercial use allowed without revenue or user caps. Check the licence text on the model card with your legal team before production use.
Which engine should serve DeepSeek-V4.1-Flash-1017 on-prem?
Use DeepSeek's vLLM/SGLang recipes for V4.1 (latest releases); prompts via the encoding folder or deepseek-recipe. Set max_tokens >= 256K per the card for long reasoning. No llama.cpp support confirmed.