Llama 4 Hardware Requirements Calculator: Size Scout and Maverick for On-Prem
This free Llama 4 hardware requirements calculator sizes GPUs, VRAM, and capital cost for running Scout or Maverick on infrastructure you own, and it is built for platform engineers and IT directors evaluating Meta's mixture-of-experts release for on-prem deployment. Enter the variant, weight precision, context length, and concurrent user count, and the tool returns model memory footprint, KV cache growth, total VRAM, GPU count, and capital cost against an equivalent cloud rental. The number that surprises most teams is the memory footprint itself: because Llama 4 is a mixture-of-experts model, every expert has to be resident in GPU memory even though only 17B parameters activate per token.
Your numbers
Both variants activate only 17B parameters per token, but every expert still has to sit in GPU memory.
FP8 is Meta's recommended production precision for Llama 4 on Hopper and Blackwell GPUs.
Scout advertises a 10M-token window and Maverick 1M, but almost no on-prem deployment serves that; plan capacity around what you will actually use.
Simultaneous in-flight requests during peak hours, not total registered users.
VRAM per GPU drives how many cards you need; enter the matching price below.
H100 80GB street price runs roughly $25,000-$32,000 in 2026; H200 and B200 run higher.
On-demand H100/H200 instances typically run $2-$6 per hour depending on provider and commitment.
Your results
Planning estimates only. Real memory use depends on serving engine, batch size, and speculative decoding settings. Benchmark on representative traffic before finalizing a purchase order.
Get your full Llama 4 sizing report
We will email you a personalized GPU and capex breakdown for Scout and Maverick side by side, plus a serving-engine configuration checklist, and a Netray AI infrastructure specialist will follow up with a benchmark plan.
No spam. Your results stay private. Unsubscribe anytime.
How Llama 4 runs on hardware you own
Scout and Maverick are both sparse mixture-of-experts models, and that architecture is the single most important fact for sizing hardware. Scout routes each token through 17B active parameters out of 109B total across 16 experts; Maverick routes the same 17B active parameters out of 400B total across 128 experts. The active-parameter count sets compute cost and, roughly, latency per token. The total-parameter count sets memory requirements, because a serving engine cannot know in advance which experts a given batch will need, so it loads them all. Practically that means Maverick at FP8 needs about 400GB of GPU memory for weights alone, before a single token of KV cache, which puts it firmly in multi-GPU or multi-node territory even at a compressed precision.
- Scout (109B total) fits on a single 80GB GPU only at aggressive INT4 quantization; FP8 needs at least two.
- Maverick (400B total) needs 5-6 H100-class GPUs at FP8 for weights alone, before KV cache overhead.
- Active parameters (17B for both variants) govern throughput and per-token latency, not memory.
- Long-context serving (beyond 128K) multiplies KV cache fast and is rarely practical outside multi-node clusters.
Quantization tradeoffs for Llama 4
FP8 is the precision Meta targeted for production serving on Hopper and Blackwell hardware and is the safest default: benchmarks generally show under 1% quality loss versus BF16 while halving memory. INT4 methods like AWQ compress further, roughly a quarter of BF16 size, and make Scout viable on a single 80GB card, but expect measurable degradation on multi-step reasoning and long tool-use chains, the exact workloads MoE routing was designed to help with. Test any quantized checkpoint against your own evaluation set rather than a public leaderboard score, because routing behavior can shift slightly under aggressive quantization and change which expert actually answers a given prompt.
- FP8 is the recommended default: near-lossless quality, roughly half the memory of BF16.
- INT4/AWQ roughly quarters memory but degrades reasoning-heavy and tool-calling tasks more than simple Q&A.
- Mixed precision (FP8 experts, BF16 attention and router) is common in production vLLM and SGLang configs.
- Re-validate routing quality after quantization; expert selection can shift under compression.
Serving Llama 4 with vLLM and SGLang
Both vLLM and SGLang shipped native Llama 4 support with expert-parallel and tensor-parallel execution, which is what makes multi-GPU serving of a 400B-total-parameter model practical. Expert parallelism spreads the MoE layers across GPUs so no single card has to hold every expert, while tensor parallelism splits the dense attention layers. Continuous batching then lets the serving engine interleave many concurrent requests against the same loaded weights, which is the only way the economics of a multi-GPU deployment work: idle GPU cycles between tokens for one user get filled with work for another. Plan your GPU topology (NVLink versus PCIe, single node versus multi-node) around your chosen serving engine's parallelism support before ordering hardware.
- vLLM and SGLang both support expert-parallel MoE serving as of their 2025-2026 releases.
- NVLink or NVSwitch interconnect matters more for MoE models than dense models due to cross-expert traffic.
- Continuous batching is what makes the capital cost of large MoE deployments defensible at scale.
- Multi-node serving (Maverick at full context, high concurrency) requires InfiniBand or a comparable low-latency fabric.
When to fine-tune Llama 4 instead of prompting
Fine-tune Scout or Maverick when you need consistent structured output, domain-specific tool-calling behavior, or a house style that prompting cannot reliably enforce across thousands of daily calls. LoRA and QLoRA are the practical entry points; full fine-tuning of a 400B-total-parameter MoE model is a serious undertaking that most enterprises should not attempt without a dedicated ML infrastructure team. A more common and cheaper path is fine-tuning Scout, since its smaller total footprint makes training iteration dramatically faster, then evaluating whether Maverick's extra capacity is actually needed for your task. Many enterprise workloads, particularly ERP-grounded question answering, do not benefit from Maverick's scale once the model has been adapted.
How Netray deploys and customizes Llama 4 on-prem
Netray deploys Llama 4 inside customer networks for aerospace, defense, and electronics manufacturers who cannot send prompts or documents to a public API. We benchmark Scout against Maverick on your actual traffic before recommending either, since the extra memory and cost of Maverick is only worth it when evaluation results prove it. We then handle the full stack: expert-parallel serving configuration in vLLM or SGLang, quantization validated against your golden question set, and integration with SyteLine, LN, M3, or ServiceMax so the model answers from live operational data. Engagements typically start with a two-week sizing and benchmark phase using your own prompts.
Frequently Asked Questions
Why does Maverick need so much more memory than a 17B model would suggest?
Because Maverick is a mixture-of-experts model with 128 experts totaling 400B parameters, and a serving engine cannot predict which experts an incoming batch of tokens will need. Every expert has to be loaded into GPU memory at all times so routing can select the right one per token. The 17B active-parameter figure describes compute cost per token, not memory footprint. This is the most common sizing mistake teams make with MoE models: budgeting GPUs as if it were a dense 17B model.
Can I run Llama 4 Scout on a single GPU?
Only at aggressive quantization. Scout's 109B total parameters need about 109GB at FP8, which exceeds a single 80GB card once KV cache is added. At INT4/AWQ, weights drop to roughly 55GB, which fits an H100 or H200 with modest concurrency and context length. For any meaningful concurrent user count, plan for two GPUs even with Scout at INT4.
Is Llama 4 worth the hardware cost compared to a dense 70B model?
It depends on the task. Llama 4's MoE architecture gives it stronger reasoning and longer effective context handling than a similarly-priced dense model, but the memory cost is substantially higher for Maverick specifically. For straightforward extraction, classification, or single-turn ERP question answering, a fine-tuned dense model like Llama 3.3 70B often matches Llama 4 quality at a fraction of the GPU count. Benchmark both against your evaluation set before committing capital.
What GPU interconnect do I need for Llama 4 Maverick?
NVLink or NVSwitch within a node is strongly recommended over PCIe-only configurations, because expert-parallel MoE serving generates significant cross-GPU traffic as tokens get routed to experts that may live on different cards. PCIe-only setups will work but throughput suffers noticeably under concurrent load. If you plan to scale beyond a single 8-GPU node, budget for InfiniBand between nodes as well.
Get a benchmarked Llama 4 deployment plan sized to your concurrency, context, and compliance requirements.
Related Tools
Qwen3 Deployment Sizing Calculator
Size VRAM, GPU count, and capital cost across the Qwen3 family, from the 235B-A22B mixture-of-experts flagship down to the dense 8B model.
On-Prem AIgpt-oss Hardware Sizing Calculator
Size VRAM, GPU count, and capital cost for OpenAI's Apache 2.0 licensed gpt-oss-20b or gpt-oss-120b, both natively quantized to MXFP4 for single-GPU deployment.
On-Prem AIGPU Sizing Calculator for LLM Inference
Work out how many GPUs you need to serve a given open-weight model to your user base, based on memory footprint and token throughput.
Go Deeper
Deploying Llama 4 On-Prem: An Enterprise Guide
Deploy Llama 4 Scout or Maverick on-prem: architecture, license terms, GPU sizing at FP8/INT4, vLLM setup, fine-tuning, and when it beats the alternatives.
How Netray Evaluates Every New AI Model in 48 Hours
Netray's repeatable framework for evaluating any new open-weight AI model release within 48 hours: license, hardware fit, benchmarks, and deployment risk.
Enterprise GPU Cluster Planning for AI Workloads
Plan an enterprise GPU cluster for AI workloads: H100 vs L40S sizing, networking, power, cooling, and cost models for on-prem LLM inference and training.