On-prem models / StepFun
Run Step-3.7-Flash on-prem
198B sparse MoE vision-language model (196B language backbone plus 1.8B vision encoder, about 11B active) built for high-throughput agent workloads. Official FP8, NVFP4 and GGUF variants exist.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 396 GB | 0.47 GB | 1.68 GB | 6.51 GB |
| FP8 | 198 GB | 0.24 GB | 0.84 GB | 3.26 GB |
| INT4 (AWQ / GPTQ) | 111 GB | 0.24 GB | 0.84 GB | 3.26 GB |
| GGUF Q4_K_M | 119 GB | 0.24 GB | 0.84 GB | 3.26 GB |
KV cache from the published config: 45 layers, 8 KV heads x 128 dims, 33 sliding-window layers (512 tokens). 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Good fit for document-heavy finance and procurement agents (reports, invoices, POs) that need vision plus tool use under a permissive licence. NVFP4 build makes a 4-GPU Blackwell deployment plausible.
Strengths
- Apache 2.0
- Broad engine coverage including official llama.cpp fork and GGUF
- Sliding-window layers keep KV small
- Card highlights parsing large financial reports and multi-step search agents
Watch out for
- About 400 GB in BF16; FP8 or NVFP4 recommended
- Some features need StepFun's vLLM image or SGLang dev build
- Smaller community than Qwen/GLM
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 2 | 115 / 115 GB | 138 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 1 | 113 / 115 GB | 189 |
| NVIDIA DGX Spark (GB10) 128GB | 1 | 113 / 115 GB | 19 |
| AMD Ryzen AI Max+ 395 128GB system | 1 | 113 / 115 GB | 18 |
| NVIDIA H200 SXM 141GB | 1 | 113 / 126 GB | 226 |
| NVIDIA H200 NVL 141GB | 1 | 113 / 126 GB | 226 |
| NVIDIA A100 40GB PCIe | 4 | 118 / 144 GB | 168 |
| NVIDIA A100 80GB PCIe | 2 | 115 / 144 GB | 156 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| AMD Instinct MI210 64GB | 4 | 213 / 230 GB | 102 |
| AMD Instinct MI325X 256GB | 1 | 209 / 230 GB | 136 |
| Intel Gaudi 3 128GB (HL-325L OAM) | 2 | 210 / 230 GB | 130 |
| NVIDIA B300 (Blackwell Ultra) 270GB | 1 | 209 / 243 GB | 169 |
| NVIDIA H200 SXM 141GB | 2 | 210 / 253 GB | 154 |
| NVIDIA H200 NVL 141GB | 2 | 210 / 253 GB | 154 |
| AMD Instinct MI355X 288GB | 1 | 209 / 259 GB | 169 |
| NVIDIA A100 40GB PCIe | 8 | 219 / 288 GB | 129 |
Serving and licence
vLLM (vllm/vllm-openai:stepfun37 image, --tool-call-parser step3p5, --reasoning-parser step3p5), SGLang dev image, Transformers, and llama.cpp via stepfun-ai fork with official GGUF; NVFP4 needs modelopt quantisation and FP8 KV cache.
Apache-2.0. Apache 2.0: commercial use, modification and redistribution allowed with notice.
Design a Step-3.7-Flash deployment
Preloaded with Step-3.7-Flash. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does Step-3.7-Flash need?
The weights alone take about 396 GB in BF16, 198 GB in FP8 and 111 GB in 4-bit. On top of that each request holds a KV cache of about 0.24 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 4.72 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs Step-3.7-Flash?
For a single user at 8K context in 4-bit, 2 x AMD Instinct MI210 64GB (64 GB each) fits, at an estimated 138 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is Step-3.7-Flash allowed for commercial use?
Step-3.7-Flash is released under Apache-2.0. Apache 2.0: commercial use, modification and redistribution allowed with notice. Check the licence text on the model card with your legal team before production use.
Which engine should serve Step-3.7-Flash on-prem?
vLLM (vllm/vllm-openai:stepfun37 image, --tool-call-parser step3p5, --reasoning-parser step3p5), SGLang dev image, Transformers, and llama.cpp via stepfun-ai fork with official GGUF; NVFP4 needs modelopt quantisation and FP8 KV cache.