Qwen3 Deployment Sizing Calculator: MoE Flagship and Dense Variants
This free Qwen3 deployment sizing calculator estimates GPU count, VRAM, and capital cost across the entire Qwen3 family, and it is built for AI platform teams choosing between the 235B-A22B mixture-of-experts flagship and the dense 32B, 14B, or 8B variants. Enter the variant, precision, context length, and concurrent user count, and the tool returns weight footprint, KV cache, total VRAM, GPU count, and cost. Qwen3's release strategy of shipping one strong MoE flagship alongside a full dense lineup at the same quality bar makes it one of the more flexible open-weight families for enterprise sizing decisions, because you can genuinely trade capability for hardware cost along a single family rather than switching model providers.
Your numbers
Only the 235B-A22B variant is mixture-of-experts; the 32B, 14B, and 8B variants are dense and activate every parameter.
Qwen3 ships official FP8 checkpoints alongside BF16; FP8 is the practical default for production serving.
Qwen3 natively supports 32K and extends to 128K with YaRN scaling; only enable YaRN if you actually need it.
Simultaneous in-flight requests during peak hours.
VRAM per GPU; enter a matching price below.
L40S runs $8,000-$11,000; H100 80GB runs roughly $25,000-$32,000 in 2026.
On-demand rates vary by GPU class and provider; H100 typically runs $2-$6 per hour.
Your results
Planning estimates only. Real memory use depends on serving engine, batch size, and YaRN context extension settings. Benchmark on representative traffic before finalizing a purchase order.
Get your full Qwen3 variant comparison
We will email you a side-by-side sizing and cost breakdown across the Qwen3 family for your workload, and a Netray AI specialist will follow up with a benchmark and deployment plan.
No spam. Your results stay private. Unsubscribe anytime.
How Qwen3 runs on hardware you own
Qwen3-235B-A22B routes 22B active parameters per token through 128 experts, of which roughly 8 activate per token, but every one of the 235B total parameters must be resident in GPU memory regardless of routing. That means at FP8 the model needs about 235GB for weights alone, comparable to a dense 235B model in memory terms even though its compute cost per token resembles a 22B model. The dense variants have no such split: a 32B dense model needs exactly 32B parameters worth of memory and delivers exactly 32B parameters worth of compute per token, which makes sizing far more predictable and is why many enterprises default to Qwen3-32B for production unless evaluation results specifically justify the MoE flagship.
- 235B-A22B needs roughly 235GB at FP8 for weights, requiring 3-4 H100-class GPUs before KV cache.
- Qwen3-32B dense fits on a single 80GB GPU at FP8 with meaningful concurrency headroom.
- Qwen3-8B dense runs comfortably on an L40S 48GB or even a workstation GPU at INT4.
- Active parameters (22B for the MoE flagship) predict latency and throughput; total parameters predict memory.
Quantization tradeoffs for Qwen3
Qwen3's official FP8 checkpoints are close to lossless against BF16 on published benchmarks and are the sensible default for any production deployment, including the dense variants. INT4/AWQ quantization compresses further and is genuinely practical for the smaller dense models (8B, 14B) where the accuracy hit is well tolerated by most extraction and classification tasks. For the 235B-A22B flagship, INT4 quantization of a mixture-of-experts model is less battle-tested than for dense models; validate router behavior carefully, since aggressive quantization can shift which experts get selected for a given input and change output quality in ways that are not visible from aggregate benchmark scores.
- FP8 is close to lossless for both the MoE flagship and the dense variants; use it as your default.
- INT4/AWQ is well-proven for the dense 8B-32B variants and enables single-GPU deployment.
- INT4 quantization of the MoE flagship needs task-specific validation against your own evaluation set.
- Thinking-mode responses (Qwen3 supports an explicit reasoning mode) generate longer outputs; budget output tokens accordingly.
Serving Qwen3 with vLLM and SGLang
Qwen3 has first-class support in both vLLM and SGLang, including expert-parallel serving for the MoE flagship and standard tensor-parallel serving for the dense variants. Qwen3's dual reasoning modes (thinking and non-thinking) matter for capacity planning: thinking mode produces substantially longer outputs before the final answer, which increases decode time and effective token cost per request even though the model itself is unchanged. Set the reasoning mode deliberately per workload rather than leaving it on the model default, and monitor output token counts in production, since a support-desk workload that silently drifts into thinking mode for every query will consume far more GPU-seconds than expected.
- vLLM and SGLang both support Qwen3's MoE and dense variants with continuous batching.
- Thinking mode meaningfully increases output length and per-request GPU time; enable it selectively.
- Tensor parallelism across 2-4 GPUs is standard for the dense 32B variant at FP8.
- Expert-parallel serving for the 235B-A22B flagship benefits from NVLink-connected GPUs.
When to fine-tune Qwen3 instead of prompting
Qwen3-8B and Qwen3-14B are strong fine-tuning targets for narrow enterprise tasks: LoRA on either trains quickly, deploys on modest hardware, and frequently matches or beats a much larger prompted model on a single well-defined task like ERP field extraction or work-order classification. Fine-tune the 235B-A22B flagship only when the task genuinely needs its broader reasoning capacity and you have the infrastructure to iterate on a model of that size; QLoRA makes this more tractable but training cycles are still measured in days rather than hours. A common and effective pattern is prototyping the task against the MoE flagship with prompting, then fine-tuning a dense 14B or 32B model once the task definition is proven.
How Netray deploys and customizes Qwen3 on-prem
Netray evaluates the full Qwen3 family against customer workloads before recommending a variant, because the right choice is almost never obvious from published benchmarks alone. For manufacturers running SyteLine, LN, or M3, we frequently land on a fine-tuned Qwen3-14B or 32B for the bulk of production traffic, reserving the 235B-A22B flagship for a narrow set of genuinely hard reasoning tasks routed through a model gateway. We handle serving configuration in vLLM or SGLang, quantization validation, and the ERP integration layer end to end. Engagements typically start with a two-week benchmark against your own prompts and documents.
Frequently Asked Questions
Should I default to the MoE flagship or a dense Qwen3 variant?
Start with a dense variant unless you have evidence you need more. Qwen3-32B dense matches or beats the 235B-A22B flagship on many enterprise tasks at a fraction of the memory footprint and with far more predictable sizing. Reserve the MoE flagship for workloads where evaluation results show a genuine quality gap, typically complex multi-step reasoning or broad general knowledge tasks rather than narrow domain-specific extraction or classification.
Does thinking mode change how I should size hardware?
Yes. Qwen3's thinking mode produces a visible reasoning trace before the final answer, often several hundred to a few thousand extra tokens. That directly increases decode time and GPU-seconds per request, which changes your effective concurrency capacity even though the model's memory footprint is unchanged. Size KV cache and throughput assumptions around whichever mode your workload will actually use in production, not the shorter non-thinking baseline.
Can Qwen3-8B run on a single consumer GPU?
Yes, comfortably. At INT4 quantization, Qwen3-8B needs roughly 4GB for weights, which fits easily on a 24GB consumer card with substantial headroom for KV cache and moderate concurrency. This makes it a practical choice for edge deployment, development environments, or air-gapped sites where a full datacenter GPU is not available.
How does Qwen3-235B-A22B compare to DeepSeek V3 for on-prem sizing?
Both are large MoE models requiring the full total parameter count in memory, but Qwen3-235B-A22B is roughly a third the total size of DeepSeek V3's 671B, making it meaningfully cheaper to deploy on-prem while remaining competitive on many benchmarks. If your hardware budget is the binding constraint, Qwen3's flagship is usually the more accessible starting point among current large MoE open-weight models.
Get a Qwen3 variant recommendation and sizing plan benchmarked against your own workloads.
Related Tools
Llama 4 Hardware Requirements Calculator
Estimate VRAM, GPU count, and capital cost to run Llama 4 Scout or Maverick on your own hardware, accounting for full mixture-of-experts weight loading and KV cache growth.
On-Prem AILlama 3.3 70B Inference Calculator
Size VRAM, GPU count, and capital cost to self-host Llama 3.3 70B, a dense model that remains the default starting point for enterprise on-prem AI in 2026.
On-Prem AIOpen-Weight Model Selector
A 10-question assessment that matches your hardware budget, workload complexity, and operational maturity to the right open-weight model size class.
Go Deeper
Qwen3 Enterprise Deployment: The On-Prem Guide
Deploy Qwen3 on-prem: MoE and dense sizes from 0.6B to 235B, Apache 2.0 license, GPU sizing by quantization, serving setup, fine-tuning, and when to use it.
The 2026 Open-Weight LLM Landscape: A Practical Map
A practical map of the 2026 open-weight LLM landscape: model families, license terms, and which model fits your VRAM budget and use case.
How Netray Evaluates Every New AI Model in 48 Hours
Netray's repeatable framework for evaluating any new open-weight AI model release within 48 hours: license, hardware fit, benchmarks, and deployment risk.