On-prem models / Moonshot AI
Run Kimi-K2.7-Code on-prem
Coding-focused post-train of Kimi K2.6 with the same 1T/32B MLA architecture. The card reports about 30% fewer thinking tokens than K2.6 on coding work.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 2000 GB | 0.58 GB | 2.30 GB | 9.21 GB |
| FP8 | 1000 GB | 0.29 GB | 1.15 GB | 4.61 GB |
| INT4 (AWQ / GPTQ) | 560 GB | 0.29 GB | 1.15 GB | 4.61 GB |
| GGUF Q4_K_M | 600 GB | 0.29 GB | 1.15 GB | 4.61 GB |
KV cache from the published config: 61 layers, MLA latent 512 + 64 RoPE dims. 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Relevant for ERP engineering teams building customisations, integrations and data pipelines with an on-prem coding agent. Not aimed at end-user document or chat workloads.
Strengths
- Strong on real-world long-horizon software tasks
- Interleaved thinking with multi-step tool calls
- Native INT4 weights
- Same serving footprint as K2.6
Watch out for
- Preserved thinking is forced and cannot be disabled
- Coding specialist; general business tasks better served by K2.6
- Needs an 8-GPU server
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA A100 80GB PCIe | 8 | 573 / 576 GB | 119 |
| NVIDIA A100 80GB SXM | 8 | 573 / 576 GB | 121 |
| NVIDIA H100 PCIe 80GB | 8 | 573 / 576 GB | 121 |
| NVIDIA H100 SXM 80GB | 8 | 573 / 576 GB | 142 |
| NVIDIA B200 180GB | 4 | 567 / 648 GB | 185 |
| NVIDIA H100 NVL 94GB | 8 | 573 / 676 GB | 148 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition 96GB | 8 | 573 / 691 GB | 110 |
| AMD Instinct MI300X 192GB | 4 | 567 / 691 GB | 162 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI355X 288GB | 4 | 1018 / 1036 GB | 142 |
| NVIDIA B200 180GB | 8 | 1024 / 1296 GB | 147 |
| AMD Instinct MI300X 192GB | 8 | 1024 / 1382 GB | 131 |
| AMD Instinct MI325X 256GB | 8 | 1024 / 1843 GB | 136 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 8 | 1024 / 1944 GB | 147 |
Serving and licence
Same engines as K2.6 (vLLM, SGLang, KTransformers); pass full assistant messages including reasoning back in multi-turn calls.
Modified MIT. MIT terms plus one condition: products over 100M MAU or US$20M monthly revenue must display 'Kimi K2.7 Code' in the UI.
Design a Kimi-K2.7-Code deployment
Preloaded with Kimi-K2.7-Code. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does Kimi-K2.7-Code need?
The weights alone take about 2000 GB in BF16, 1000 GB in FP8 and 560 GB in 4-bit. On top of that each request holds a KV cache of about 0.29 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 5.76 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs Kimi-K2.7-Code?
For a single user at 8K context in 4-bit, 8 x NVIDIA A100 80GB PCIe (80 GB each) fits, at an estimated 119 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is Kimi-K2.7-Code allowed for commercial use?
Kimi-K2.7-Code is released under Modified MIT. MIT terms plus one condition: products over 100M MAU or US$20M monthly revenue must display 'Kimi K2.7 Code' in the UI. Check the licence text on the model card with your legal team before production use.
Which engine should serve Kimi-K2.7-Code on-prem?
Same engines as K2.6 (vLLM, SGLang, KTransformers); pass full assistant messages including reasoning back in multi-turn calls.