# gpt-oss Hardware Sizing Calculator: OpenAI's Open-Weight Models On-Prem

> Source: https://netray.co/tools/gpt-oss-hardware-sizing-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Size VRAM, GPU count, and capital cost for OpenAI's Apache 2.0 licensed gpt-oss-20b or gpt-oss-120b, both natively quantized to MXFP4 for single-GPU deployment.

## Category

on-prem-ai

## Tool

**Kind:** calculator

**Inputs:**

#### gpt-oss variant

**Key:** model_total_params_b

**Type:** select

**Default:** 117

**Unit:** B params

**Help:** Both variants ship natively in MXFP4 and activate a small fraction of total parameters per token via MoE routing.

**Options:**

##### gpt-oss-120b: 117B total, 5.1B active MoE

**Value:** 117

##### gpt-oss-20b: 21B total, 3.6B active MoE

**Value:** 21

#### Weight precision

**Key:** quant_bytes

**Type:** select

**Default:** 0.5

**Unit:** bytes/param

**Help:** MXFP4 is gpt-oss's native release format, roughly 4 bits per parameter, and the recommended default.

**Options:**

##### MXFP4, native (0.5 bytes/param)

**Value:** 0.5

##### FP8, dequantized (1 byte/param)

**Value:** 1

##### FP16 / BF16, dequantized (2 bytes/param)

**Value:** 2

#### Serving context length

**Key:** context_tokens

**Type:** slider

**Default:** 32768

**Min:** 4096

**Max:** 131072

**Step:** 4096

**Unit:** tokens

**Help:** Both variants support up to 128K context.

#### Concurrent active users

**Key:** concurrent_users

**Type:** number

**Default:** 20

**Min:** 1

**Max:** 300

**Step:** 1

**Unit:** users

**Help:** Simultaneous in-flight requests during peak hours.

#### GPU type

**Key:** gpu_vram_gb

**Type:** select

**Default:** 80

**Unit:** GB VRAM

**Help:** VRAM per GPU; enter a matching price below.

**Options:**

##### RTX 4090 24GB

**Value:** 24

##### L40S 48GB

**Value:** 48

##### A100 80GB SXM

**Value:** 80

##### H100 80GB SXM

**Value:** 80

#### Price per GPU

**Key:** gpu_price

**Type:** number

**Default:** 30000

**Min:** 500

**Max:** 70000

**Step:** 500

**Unit:** $

**Help:** H100 80GB runs roughly $25,000-$32,000; RTX 4090 runs $1,600-$2,200 for smaller deployments.

#### Cloud rental rate per equivalent GPU

**Key:** cloud_hourly_rate

**Type:** number

**Default:** 3

**Min:** 0.2

**Max:** 10

**Step:** 0.1

**Unit:** $/hr

**Help:** On-demand rates vary by GPU class; H100 typically runs $2-$6 per hour.

**Outputs:**

#### Model weights footprint

**Key:** model_weights_gb

**Expr:** model_total_params_b * quant_bytes

**Format:** number

**Description:** Full expert set at your chosen precision; only the active-parameter subset computes per token, but all experts must be resident.

#### KV cache at peak concurrency

**Key:** kv_cache_gb

**Expr:** concurrent_users * context_tokens / 1000 * 0.2

**Format:** number

**Description:** gpt-oss uses grouped attention with alternating dense and sliding-window layers to keep cache growth moderate.

#### Total VRAM required

**Key:** total_vram_gb

**Expr:** model_weights_gb + kv_cache_gb + model_weights_gb * 0.05

**Format:** number

**Description:** Weights plus KV cache plus a 5% allowance for runtime overhead.

**Primary:** true

#### GPUs required

**Key:** gpus_required

**Expr:** ceil(total_vram_gb / gpu_vram_gb)

**Format:** number

**Description:** Minimum accelerator count for one serving replica at your target concurrency.

#### Hardware capital cost

**Key:** hardware_capex

**Expr:** gpus_required * gpu_price

**Format:** currency

**Description:** One-time GPU spend at your entered price per card.

**Primary:** true

#### Equivalent monthly cloud rental

**Key:** monthly_cloud_cost

**Expr:** gpus_required * cloud_hourly_rate * 730

**Format:** currency

**Description:** What the same GPU count would cost per month rented on-demand, for capex payback comparison.

**Disclaimer:** Planning estimates only. Real memory use depends on serving engine support for native MXFP4 kernels. Benchmark on representative traffic before finalizing a purchase order.

## Content

**Intro:** This free gpt-oss hardware sizing calculator estimates GPU count, VRAM, and capital cost for OpenAI's Apache 2.0 licensed open-weight models, and it is built for teams evaluating gpt-oss-20b or gpt-oss-120b for on-prem deployment. Enter the variant, precision, context length, and concurrency, and the tool returns weight footprint, KV cache, total VRAM, GPU count, and cost. Both gpt-oss models ship natively quantized to MXFP4, a roughly 4-bit format, which means the memory savings other open-weight models only achieve through post-training quantization come built in, making gpt-oss-120b deployable on a single 80GB GPU and gpt-oss-20b deployable on a single consumer card.

**Sections:**

#### How gpt-oss runs on hardware you own

**Body:** Both gpt-oss models are mixture-of-experts architectures: gpt-oss-120b routes 5.1B active parameters per token through 117B total, and gpt-oss-20b routes 3.6B active parameters through 21B total. As with any MoE model, the full expert set must be resident in memory regardless of routing. What makes gpt-oss unusual is that OpenAI released both models natively in MXFP4, so the memory savings of aggressive quantization are the default rather than an optional post-processing step. At native MXFP4, gpt-oss-120b needs roughly 58.5GB, fitting a single 80GB H100 with room to spare, and gpt-oss-20b needs roughly 10.5GB, fitting comfortably on a single 24GB consumer GPU.

**Bullets:**

- gpt-oss-120b at native MXFP4 needs roughly 58.5GB, fitting a single 80GB GPU.
- gpt-oss-20b at native MXFP4 needs roughly 10.5GB, fitting a single 24GB consumer GPU.
- Active parameters (5.1B and 3.6B respectively) are unusually small relative to total size, keeping compute cost low.
- Apache 2.0 licensing removes the commercial-use restrictions some other open-weight models carry.

#### Quantization tradeoffs for gpt-oss

**Body:** MXFP4 is not a compromise for gpt-oss the way post-training INT4 quantization can be for other models; it is the format OpenAI trained and evaluated the released checkpoints in, so it carries the least risk of unexpected quality loss. Dequantizing to FP8 or FP16 roughly doubles or quadruples the memory footprint for a quality difference most enterprise evaluations will not detect, since the model was not further optimized at those precisions. The practical reason to consider FP8 is hardware compatibility: not every serving stack has mature MXFP4 kernel support yet, and dequantizing to a more universally supported format may be the pragmatic choice on some infrastructure.

**Bullets:**

- MXFP4 is gpt-oss's native, evaluated release format and the recommended default wherever supported.
- Dequantizing to FP8 or FP16 roughly doubles or quadruples memory for minimal quality gain.
- Choose a higher precision mainly for serving-stack compatibility, not for expected quality improvement.
- Verify your serving engine has mature native MXFP4 kernel support before committing to a hardware plan.

#### Serving gpt-oss with vLLM and SGLang

**Body:** vLLM and SGLang both added native MXFP4 support for gpt-oss shortly after release, including expert-parallel MoE serving, which is what makes single-GPU deployment of gpt-oss-120b practical despite its 117B total parameter count. Because both models are unusually small at their active-parameter level relative to total size, throughput per GPU tends to be strong once weights are loaded, closer to what you would expect from a much smaller dense model. Confirm your serving engine version explicitly supports MXFP4 kernels for your target GPU generation before finalizing a deployment plan, since kernel support has been an active area of development since release.

**Bullets:**

- vLLM and SGLang both support native MXFP4 serving with expert-parallel MoE routing.
- Strong per-GPU throughput relative to total parameter count due to small active-parameter footprint.
- Confirm MXFP4 kernel support for your specific GPU generation before committing hardware.
- Single-GPU deployment of gpt-oss-120b removes multi-GPU interconnect complexity entirely.

#### When to fine-tune gpt-oss instead of prompting

**Body:** gpt-oss-20b is an attractive fine-tuning target precisely because its small active-parameter footprint keeps training iteration fast even though 21B total parameters must be handled during the process; LoRA and QLoRA are both practical on a single workstation-class GPU. Fine-tune when you need consistent output structure or domain terminology adherence that Apache 2.0 licensing lets you deploy commercially without the licensing review some other open-weight models require. gpt-oss-120b fine-tuning needs more substantial infrastructure but remains far more accessible than a similarly-capable dense model given its small active-parameter compute cost during training.

#### How Netray deploys and customizes gpt-oss on-prem

**Body:** Netray deploys gpt-oss for manufacturers who want the permissive Apache 2.0 license alongside genuinely low hardware requirements: gpt-oss-120b's single-GPU footprint makes it one of the more capital-efficient large models we recommend for customers starting their on-prem AI program. We configure MXFP4-native serving in vLLM or SGLang, fine-tune for domain-specific tasks, and integrate outputs with SyteLine, LN, and M3. Engagements typically start with a two-week pilot proving the model against your actual documents and workflows before scaling to production.

**Cta:** Get a gpt-oss deployment plan sized for single-GPU efficiency and benchmarked on your own workloads.

**Faq:**

#### Why does gpt-oss-120b fit on a single GPU when other 100B+ models do not?

**Answer:** Because it ships natively quantized to MXFP4, a roughly 4-bit format, rather than requiring a separate post-training quantization step. At MXFP4, its 117B total parameters need only about 58.5GB of memory, comfortably under the 80GB capacity of a single H100 or A100. Most other large open-weight models are released in BF16 or FP8 first, and users have to apply their own quantization, which introduces additional accuracy risk that gpt-oss's native release avoids.

#### What does Apache 2.0 licensing actually change for enterprise use?

**Answer:** Apache 2.0 is one of the most permissive open-source licenses available, allowing commercial use, modification, and redistribution without royalty obligations or the usage restrictions some other open-weight model licenses impose above certain revenue or user thresholds. For enterprises that need legal certainty about commercial deployment without a lengthy license review, this is a meaningful practical advantage over models released under more restrictive community licenses.

#### How does gpt-oss compare to Qwen3 or Gemma 3 for single-GPU deployment?

**Answer:** gpt-oss-20b and Qwen3-8B or Gemma 3 4B occupy a similar single-GPU deployment niche, and the right choice depends on task-specific benchmark performance rather than hardware requirements alone, since all fit comfortably on a single card at appropriate quantization. gpt-oss-120b is unusual in offering flagship-scale capability on a single 80GB GPU, a footprint that models like Llama 3.3 70B or Mistral Large need two or more GPUs to match at comparable precision.

#### Does MXFP4 quantization hurt accuracy compared to a model trained in BF16?

**Answer:** OpenAI evaluated and published benchmark results for the MXFP4 checkpoints directly, so published scores already reflect this precision rather than being measured at a higher precision and assumed to degrade under quantization. In practice this means MXFP4 carries less uncertainty than post-training quantization of a model released in a different native format, though you should still validate against your own evaluation set for domain-specific tasks.

## Search terms

gpt-oss hardware sizing, gpt-oss-120b GPU requirements, gpt-oss-20b deployment, on-prem gpt-oss, self-hosted gpt-oss, MXFP4 quantization, OpenAI open weight model, gpt-oss fine-tuning

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json