On-prem models / Alibaba Qwen
Run Qwen3-Coder-Next on-prem
Coding-agent model on the Qwen3-Next architecture: 80B total, 3B active, non-thinking only. Built for use inside CLI and IDE agents such as Qwen Code, Cline and Claude Code style harnesses.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 159 GB | 0.20 GB | 0.81 GB | 3.22 GB |
| FP8 | 79.7 GB | 0.10 GB | 0.40 GB | 1.61 GB |
| INT4 (AWQ / GPTQ) | 44.6 GB | 0.10 GB | 0.40 GB | 1.61 GB |
| GGUF Q4_K_M | 47.8 GB | 0.10 GB | 0.40 GB | 1.61 GB |
KV cache from the published config: 48 layers, 2 KV heads x 256 dims, 12 full-attention layers. 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Useful for ERP development teams: writing and refactoring customisations, reports, integration scripts and SQL against a schema. Less suited to end-user document extraction, which needs a vision model.
Strengths
- Strong agentic coding and tool use at 3B active
- Fits 2x48 GB or one 80 GB GPU in FP8/4-bit
- 256K context for repository-scale work
- Apache 2.0
Watch out for
- Non-thinking only
- Text only, no vision
- About 160 GB in BF16
- Tuned for code; general business writing is not its focus
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA GeForce RTX 5090 32GB | 2 | 48 / 57 GB | 274 |
| AMD Instinct MI210 64GB | 1 | 47 / 57 GB | 256 |
| NVIDIA A100 40GB PCIe | 2 | 48 / 72 GB | 261 |
| NVIDIA A100 80GB PCIe | 1 | 47 / 72 GB | 286 |
| NVIDIA A100 80GB SXM | 1 | 47 / 72 GB | 295 |
| NVIDIA H100 PCIe 80GB | 1 | 47 / 72 GB | 292 |
| NVIDIA H100 SXM 80GB | 1 | 47 / 72 GB | 377 |
| NVIDIA H100 NVL 94GB | 1 | 47 / 84 GB | 402 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 1 | 86 / 86 GB | 128 |
| NVIDIA RTX PRO 6000 Blackwell Max-Q 96GB | 1 | 86 / 86 GB | 128 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition 96GB | 1 | 86 / 86 GB | 117 |
| NVIDIA GeForce RTX 5090 32GB | 4 | 90 / 115 GB | 190 |
| AMD Instinct MI210 64GB | 2 | 87 / 115 GB | 157 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 1 | 86 / 115 GB | 220 |
| NVIDIA DGX Spark (GB10) 128GB | 1 | 86 / 115 GB | 23 |
| AMD Ryzen AI Max+ 395 128GB system | 1 | 86 / 115 GB | 22 |
Serving and licence
Card requires sglang>=0.5.8 or vllm>=0.15.0 with --tool-call-parser qwen3_coder; official GGUF repo (Qwen/Qwen3-Coder-Next-GGUF) for llama.cpp/Ollama/LM Studio.
Apache-2.0. Apache 2.0: commercial use, modification and redistribution allowed with notice; no usage caps.
Design a Qwen3-Coder-Next deployment
Preloaded with Qwen3-Coder-Next. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does Qwen3-Coder-Next need?
The weights alone take about 159 GB in BF16, 79.7 GB in FP8 and 44.6 GB in 4-bit. On top of that each request holds a KV cache of about 0.10 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 2.01 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs Qwen3-Coder-Next?
For a single user at 8K context in 4-bit, 2 x NVIDIA GeForce RTX 5090 32GB (32 GB each) fits, at an estimated 274 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is Qwen3-Coder-Next allowed for commercial use?
Qwen3-Coder-Next is released under Apache-2.0. Apache 2.0: commercial use, modification and redistribution allowed with notice; no usage caps. Check the licence text on the model card with your legal team before production use.
Which engine should serve Qwen3-Coder-Next on-prem?
Card requires sglang>=0.5.8 or vllm>=0.15.0 with --tool-call-parser qwen3_coder; official GGUF repo (Qwen/Qwen3-Coder-Next-GGUF) for llama.cpp/Ollama/LM Studio.