# Llama 4 Hardware Requirements Calculator: Size Scout and Maverick for On-Prem

> Source: https://netray.co/tools/llama-4-hardware-requirements-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Estimate VRAM, GPU count, and capital cost to run Llama 4 Scout or Maverick on your own hardware, accounting for full mixture-of-experts weight loading and KV cache growth.

## Category

on-prem-ai

## Tool

**Kind:** calculator

**Inputs:**

#### Llama 4 variant

**Key:** model_total_params_b

**Type:** select

**Default:** 400

**Unit:** B params

**Help:** Both variants activate only 17B parameters per token, but every expert still has to sit in GPU memory.

**Options:**

##### Llama 4 Scout: 109B total, 17B active MoE, 16 experts

**Value:** 109

##### Llama 4 Maverick: 400B total, 17B active MoE, 128 experts

**Value:** 400

#### Weight precision

**Key:** quant_bytes

**Type:** select

**Default:** 1

**Unit:** bytes/param

**Help:** FP8 is Meta's recommended production precision for Llama 4 on Hopper and Blackwell GPUs.

**Options:**

##### FP16 / BF16 (2 bytes/param)

**Value:** 2

##### FP8 (1 byte/param)

**Value:** 1

##### INT4 / AWQ (0.5 bytes/param)

**Value:** 0.5

#### Serving context length

**Key:** context_tokens

**Type:** slider

**Default:** 32768

**Min:** 4096

**Max:** 131072

**Step:** 4096

**Unit:** tokens

**Help:** Scout advertises a 10M-token window and Maverick 1M, but almost no on-prem deployment serves that; plan capacity around what you will actually use.

#### Concurrent active users

**Key:** concurrent_users

**Type:** number

**Default:** 20

**Min:** 1

**Max:** 300

**Step:** 1

**Unit:** users

**Help:** Simultaneous in-flight requests during peak hours, not total registered users.

#### GPU type

**Key:** gpu_vram_gb

**Type:** select

**Default:** 80

**Unit:** GB VRAM

**Help:** VRAM per GPU drives how many cards you need; enter the matching price below.

**Options:**

##### A100 80GB SXM

**Value:** 80

##### H100 80GB SXM

**Value:** 80

##### H200 141GB SXM

**Value:** 141

##### B200 180GB

**Value:** 180

#### Price per GPU

**Key:** gpu_price

**Type:** number

**Default:** 30000

**Min:** 5000

**Max:** 70000

**Step:** 500

**Unit:** $

**Help:** H100 80GB street price runs roughly $25,000-$32,000 in 2026; H200 and B200 run higher.

#### Cloud rental rate per equivalent GPU

**Key:** cloud_hourly_rate

**Type:** number

**Default:** 3.5

**Min:** 0.5

**Max:** 10

**Step:** 0.1

**Unit:** $/hr

**Help:** On-demand H100/H200 instances typically run $2-$6 per hour depending on provider and commitment.

**Outputs:**

#### Model weights footprint

**Key:** model_weights_gb

**Expr:** model_total_params_b * quant_bytes

**Format:** number

**Description:** Every expert's weights at your chosen precision, whether or not that expert is active for a given token.

#### KV cache at peak concurrency

**Key:** kv_cache_gb

**Expr:** concurrent_users * context_tokens / 1000 * 0.5

**Format:** number

**Description:** Attention cache growth across concurrent sessions at your target context length.

#### Total VRAM required

**Key:** total_vram_gb

**Expr:** model_weights_gb + kv_cache_gb + model_weights_gb * 0.05

**Format:** number

**Description:** Weights plus KV cache plus a 5% allowance for CUDA context and activation buffers.

**Primary:** true

#### GPUs required

**Key:** gpus_required

**Expr:** ceil(total_vram_gb / gpu_vram_gb)

**Format:** number

**Description:** Minimum accelerator count to hold the model and serve your peak concurrency in one replica.

#### Hardware capital cost

**Key:** hardware_capex

**Expr:** gpus_required * gpu_price

**Format:** currency

**Description:** One-time GPU spend at your entered price per card, before servers, networking, or storage.

**Primary:** true

#### Equivalent monthly cloud rental

**Key:** monthly_cloud_cost

**Expr:** gpus_required * cloud_hourly_rate * 730

**Format:** currency

**Description:** What the same GPU count would cost per month rented on-demand, for capex payback comparison.

**Disclaimer:** Planning estimates only. Real memory use depends on serving engine, batch size, and speculative decoding settings. Benchmark on representative traffic before finalizing a purchase order.

## Content

**Intro:** This free Llama 4 hardware requirements calculator sizes GPUs, VRAM, and capital cost for running Scout or Maverick on infrastructure you own, and it is built for platform engineers and IT directors evaluating Meta's mixture-of-experts release for on-prem deployment. Enter the variant, weight precision, context length, and concurrent user count, and the tool returns model memory footprint, KV cache growth, total VRAM, GPU count, and capital cost against an equivalent cloud rental. The number that surprises most teams is the memory footprint itself: because Llama 4 is a mixture-of-experts model, every expert has to be resident in GPU memory even though only 17B parameters activate per token.

**Sections:**

#### How Llama 4 runs on hardware you own

**Body:** Scout and Maverick are both sparse mixture-of-experts models, and that architecture is the single most important fact for sizing hardware. Scout routes each token through 17B active parameters out of 109B total across 16 experts; Maverick routes the same 17B active parameters out of 400B total across 128 experts. The active-parameter count sets compute cost and, roughly, latency per token. The total-parameter count sets memory requirements, because a serving engine cannot know in advance which experts a given batch will need, so it loads them all. Practically that means Maverick at FP8 needs about 400GB of GPU memory for weights alone, before a single token of KV cache, which puts it firmly in multi-GPU or multi-node territory even at a compressed precision.

**Bullets:**

- Scout (109B total) fits on a single 80GB GPU only at aggressive INT4 quantization; FP8 needs at least two.
- Maverick (400B total) needs 5-6 H100-class GPUs at FP8 for weights alone, before KV cache overhead.
- Active parameters (17B for both variants) govern throughput and per-token latency, not memory.
- Long-context serving (beyond 128K) multiplies KV cache fast and is rarely practical outside multi-node clusters.

#### Quantization tradeoffs for Llama 4

**Body:** FP8 is the precision Meta targeted for production serving on Hopper and Blackwell hardware and is the safest default: benchmarks generally show under 1% quality loss versus BF16 while halving memory. INT4 methods like AWQ compress further, roughly a quarter of BF16 size, and make Scout viable on a single 80GB card, but expect measurable degradation on multi-step reasoning and long tool-use chains, the exact workloads MoE routing was designed to help with. Test any quantized checkpoint against your own evaluation set rather than a public leaderboard score, because routing behavior can shift slightly under aggressive quantization and change which expert actually answers a given prompt.

**Bullets:**

- FP8 is the recommended default: near-lossless quality, roughly half the memory of BF16.
- INT4/AWQ roughly quarters memory but degrades reasoning-heavy and tool-calling tasks more than simple Q&A.
- Mixed precision (FP8 experts, BF16 attention and router) is common in production vLLM and SGLang configs.
- Re-validate routing quality after quantization; expert selection can shift under compression.

#### Serving Llama 4 with vLLM and SGLang

**Body:** Both vLLM and SGLang shipped native Llama 4 support with expert-parallel and tensor-parallel execution, which is what makes multi-GPU serving of a 400B-total-parameter model practical. Expert parallelism spreads the MoE layers across GPUs so no single card has to hold every expert, while tensor parallelism splits the dense attention layers. Continuous batching then lets the serving engine interleave many concurrent requests against the same loaded weights, which is the only way the economics of a multi-GPU deployment work: idle GPU cycles between tokens for one user get filled with work for another. Plan your GPU topology (NVLink versus PCIe, single node versus multi-node) around your chosen serving engine's parallelism support before ordering hardware.

**Bullets:**

- vLLM and SGLang both support expert-parallel MoE serving as of their 2025-2026 releases.
- NVLink or NVSwitch interconnect matters more for MoE models than dense models due to cross-expert traffic.
- Continuous batching is what makes the capital cost of large MoE deployments defensible at scale.
- Multi-node serving (Maverick at full context, high concurrency) requires InfiniBand or a comparable low-latency fabric.

#### When to fine-tune Llama 4 instead of prompting

**Body:** Fine-tune Scout or Maverick when you need consistent structured output, domain-specific tool-calling behavior, or a house style that prompting cannot reliably enforce across thousands of daily calls. LoRA and QLoRA are the practical entry points; full fine-tuning of a 400B-total-parameter MoE model is a serious undertaking that most enterprises should not attempt without a dedicated ML infrastructure team. A more common and cheaper path is fine-tuning Scout, since its smaller total footprint makes training iteration dramatically faster, then evaluating whether Maverick's extra capacity is actually needed for your task. Many enterprise workloads, particularly ERP-grounded question answering, do not benefit from Maverick's scale once the model has been adapted.

#### How Netray deploys and customizes Llama 4 on-prem

**Body:** Netray deploys Llama 4 inside customer networks for aerospace, defense, and electronics manufacturers who cannot send prompts or documents to a public API. We benchmark Scout against Maverick on your actual traffic before recommending either, since the extra memory and cost of Maverick is only worth it when evaluation results prove it. We then handle the full stack: expert-parallel serving configuration in vLLM or SGLang, quantization validated against your golden question set, and integration with SyteLine, LN, M3, or ServiceMax so the model answers from live operational data. Engagements typically start with a two-week sizing and benchmark phase using your own prompts.

**Cta:** Get a benchmarked Llama 4 deployment plan sized to your concurrency, context, and compliance requirements.

**Faq:**

#### Why does Maverick need so much more memory than a 17B model would suggest?

**Answer:** Because Maverick is a mixture-of-experts model with 128 experts totaling 400B parameters, and a serving engine cannot predict which experts an incoming batch of tokens will need. Every expert has to be loaded into GPU memory at all times so routing can select the right one per token. The 17B active-parameter figure describes compute cost per token, not memory footprint. This is the most common sizing mistake teams make with MoE models: budgeting GPUs as if it were a dense 17B model.

#### Can I run Llama 4 Scout on a single GPU?

**Answer:** Only at aggressive quantization. Scout's 109B total parameters need about 109GB at FP8, which exceeds a single 80GB card once KV cache is added. At INT4/AWQ, weights drop to roughly 55GB, which fits an H100 or H200 with modest concurrency and context length. For any meaningful concurrent user count, plan for two GPUs even with Scout at INT4.

#### Is Llama 4 worth the hardware cost compared to a dense 70B model?

**Answer:** It depends on the task. Llama 4's MoE architecture gives it stronger reasoning and longer effective context handling than a similarly-priced dense model, but the memory cost is substantially higher for Maverick specifically. For straightforward extraction, classification, or single-turn ERP question answering, a fine-tuned dense model like Llama 3.3 70B often matches Llama 4 quality at a fraction of the GPU count. Benchmark both against your evaluation set before committing capital.

#### What GPU interconnect do I need for Llama 4 Maverick?

**Answer:** NVLink or NVSwitch within a node is strongly recommended over PCIe-only configurations, because expert-parallel MoE serving generates significant cross-GPU traffic as tokens get routed to experts that may live on different cards. PCIe-only setups will work but throughput suffers noticeably under concurrent load. If you plan to scale beyond a single 8-GPU node, budget for InfiniBand between nodes as well.

## Search terms

Llama 4 hardware requirements, Llama 4 Maverick GPU sizing, Llama 4 Scout deployment, on-prem Llama 4, self-hosted Llama 4, Llama 4 VRAM calculator, MoE model GPU requirements, Llama 4 fine-tuning

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json