On-prem models / Xiaomi MiMo
Run MiMo-V2.6-Flash-RL on-prem
Smaller sibling of MiMo-V2.6-Pro: 309B total, 15B active, omnimodal (text, image, video, audio) with 1M context. Experts are stored in MXFP4 with FP8 attention weights.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 618 GB | 0.24 GB | 0.92 GB | 3.64 GB |
| FP8 | 309 GB | 0.12 GB | 0.46 GB | 1.82 GB |
| INT4 (AWQ / GPTQ) | 173 GB | 0.12 GB | 0.46 GB | 1.82 GB |
| GGUF Q4_K_M | 185 GB | 0.12 GB | 0.46 GB | 1.82 GB |
KV cache from the published config: 48 layers, 4 KV heads x 192 dims, 39 sliding-window layers (128 tokens). 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
A credible on-prem multimodal agent for manufacturing: extract from scanned paperwork, inspect photos, transcribe operator audio and act through ERP tools. Validate structured-output reliability before production.
Strengths
- Low KV cost at long context
- Text, image, video and audio inputs
- MIT licence
- Card shows a 4-GPU vLLM launch
Watch out for
- Needs recent SGLang/vLLM or the vendor Docker image
- MXFP4 runs best on Blackwell-class GPUs
- Less community tooling than Qwen or GLM
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 4 | 180 / 230 GB | 145 |
| AMD Instinct MI325X 256GB | 1 | 175 / 230 GB | 216 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 2 | 177 / 230 GB | 189 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 1 | 175 / 243 GB | 260 |
| NVIDIA H200 SXM 141GB | 2 | 177 / 253 GB | 216 |
| NVIDIA H200 NVL 141GB | 2 | 177 / 253 GB | 216 |
| AMD Instinct MI355X 288GB | 1 | 175 / 259 GB | 260 |
| NVIDIA A100 40GB PCIe | 8 | 186 / 288 GB | 159 |
20 people at once, 16K context, MXFP4
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 2 | 172 / 172 GB | 104 |
| NVIDIA RTX PRO 6000 Blackwell Max-Q 96GB | 2 | 172 / 172 GB | 104 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition 96GB | 2 | 172 / 172 GB | 95 |
| AMD Instinct MI300X 192GB | 1 | 170 / 172 GB | 170 |
| AMD Instinct MI210 64GB | 4 | 175 / 230 GB | 130 |
| AMD Instinct MI325X 256GB | 1 | 170 / 230 GB | 186 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 2 | 172 / 230 GB | 168 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 1 | 170 / 243 GB | 227 |
Serving and licence
SGLang MiMo cookbook (--tool-call-parser mimo) and vLLM recipe (vllm/vllm-openai:mimov25-cu129, TP 4 example).
MIT. MIT licence: commercial use allowed without revenue or user caps.
Design a MiMo-V2.6-Flash-RL deployment
Preloaded with MiMo-V2.6-Flash-RL. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does MiMo-V2.6-Flash-RL need?
The weights alone take about 618 GB in BF16, 309 GB in FP8 and 173 GB in 4-bit. On top of that each request holds a KV cache of about 0.12 GB at 8K tokens of context (MXFP4 cache), so 20 people generating at once need roughly 2.42 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs MiMo-V2.6-Flash-RL?
For a single user at 8K context in 4-bit, 4 x AMD Instinct MI210 64GB (64 GB each) fits, at an estimated 145 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is MiMo-V2.6-Flash-RL allowed for commercial use?
MiMo-V2.6-Flash-RL is released under MIT. MIT licence: commercial use allowed without revenue or user caps. Check the licence text on the model card with your legal team before production use.
Which engine should serve MiMo-V2.6-Flash-RL on-prem?
SGLang MiMo cookbook (--tool-call-parser mimo) and vLLM recipe (vllm/vllm-openai:mimov25-cu129, TP 4 example).