On-prem models / Meta
Run Llama 4 Scout 17B-16E Instruct on-prem
A 16-expert MoE with 17B active parameters and native image input. It was released in April 2025 alongside Maverick. Meta positions it for very long context on a single host.
Memory needed
| Precision | Weights | KV per request, 8K | KV per request, 32K | KV per request, 128K |
|---|---|---|---|---|
| BF16 (full precision) | 218 GB | 1.61 GB | 2.82 GB | 7.65 GB |
| FP8 | 109 GB | 0.81 GB | 1.41 GB | 3.83 GB |
| INT4 (AWQ / GPTQ) | 61.0 GB | 0.81 GB | 1.41 GB | 3.83 GB |
| GGUF Q4_K_M | 65.4 GB | 0.81 GB | 1.41 GB | 3.83 GB |
KV cache from the published config: 48 layers, 8 KV heads x 128 dims, 36 sliding-window layers (8192 tokens). 16-bit cache for BF16 rows, 8-bit otherwise.
Fit for ERP and manufacturing work
Usable for reading long manuals, contracts or multi-document packs and for image-based document extraction. Tool calling works but newer models are more reliable for multi-step ERP agents. Consider it mainly where its long-context and vision combination matters.
Strengths
- Very long advertised context with a compact KV cache on chunked layers
- Native image understanding
- Fast decode for its size with 17B active
- Fits one 80GB GPU at 4-bit per Meta guidance
Watch out for
- Community reception was mixed versus benchmark claims
- 109B total parameters must stay in memory despite 17B active
- 10M context is rarely usable in practice due to memory and quality
- No reasoning mode
GPUs that run it
One user, 8K context, 4-bit
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA A100 40GB PCIe | 2 | 65 / 72 GB | 97 |
| NVIDIA A100 80GB PCIe | 1 | 64 / 72 GB | 77 |
| NVIDIA A100 80GB SXM | 1 | 64 / 72 GB | 81 |
| NVIDIA H100 PCIe 80GB | 1 | 64 / 72 GB | 80 |
| NVIDIA H100 SXM 80GB | 1 | 64 / 72 GB | 123 |
| NVIDIA H100 NVL 94GB | 1 | 64 / 84 GB | 140 |
| NVIDIA L4 24GB | 4 | 68 / 86 GB | 40 |
| NVIDIA A10 24GB | 4 | 68 / 86 GB | 70 |
20 people at once, 16K context, FP8
| Hardware | Count | Memory used | Tok/s per user |
|---|---|---|---|
| NVIDIA A100 40GB PCIe | 4 | 136 / 144 GB | 58 |
| NVIDIA A100 80GB PCIe | 2 | 133 / 144 GB | 42 |
| NVIDIA A100 80GB SXM | 2 | 133 / 144 GB | 44 |
| NVIDIA H100 PCIe 80GB | 2 | 133 / 144 GB | 44 |
| NVIDIA H100 SXM 80GB | 2 | 133 / 144 GB | 68 |
| NVIDIA B200 180GB | 1 | 131 / 162 GB | 93 |
| NVIDIA H100 NVL 94GB | 2 | 133 / 169 GB | 77 |
| NVIDIA L4 24GB | 8 | 142 / 172 GB | 23 |
Serving and licence
vLLM (with Llama 4 support from 0.8.3), SGLang, TensorRT-LLM and llama.cpp support it; Ollama offers llama4:scout. Set max-model-len to what memory allows.
Llama 4 Community License. Commercial use is allowed, but companies with over 700 million monthly active users need a separate licence from Meta, and attribution plus the Acceptable Use Policy apply.
Design a Llama 4 Scout 17B-16E Instruct deployment
Preloaded with Llama 4 Scout 17B-16E Instruct. Add your ERP and documents, change GPUs or users, and export the diagram.
Questions
How much VRAM does Llama 4 Scout 17B-16E Instruct need?
The weights alone take about 218 GB in BF16, 109 GB in FP8 and 61.0 GB in 4-bit. On top of that each request holds a KV cache of about 0.81 GB at 8K tokens of context (FP8 cache), so 20 people generating at once need roughly 16.1 GB more, plus a few GB of runtime overhead.
What is the smallest GPU setup that runs Llama 4 Scout 17B-16E Instruct?
For a single user at 8K context in 4-bit, 2 x NVIDIA A100 40GB PCIe (40 GB each) fits, at an estimated 97 tokens per second. For a team, size for concurrent requests and context; the builder above does that.
Is Llama 4 Scout 17B-16E Instruct allowed for commercial use?
Llama 4 Scout 17B-16E Instruct is released under Llama 4 Community License. Commercial use is allowed, but companies with over 700 million monthly active users need a separate licence from Meta, and attribution plus the Acceptable Use Policy apply. Check the licence text on the model card with your legal team before production use.
Which engine should serve Llama 4 Scout 17B-16E Instruct on-prem?
vLLM (with Llama 4 support from 0.8.3), SGLang, TensorRT-LLM and llama.cpp support it; Ollama offers llama4:scout. Set max-model-len to what memory allows.