GPUs for on-prem AI / NVIDIA
NVIDIA B300 (Blackwell Ultra) 270GB for LLMs
Blackwell Ultra in HGX/DGX B300 8-GPU systems (shipping, about 14 kW per DGX), with 13.5 PFLOPS dense FP4 per GPU and 2x B200 attention throughput. 270GB per GPU is derived from NVIDIA's 2.1 TB per 8 GPUs; BF16/FP8 are NVIDIA sparse totals halved; 1100W per Lenovo SR680a V4 certified listing. Sold only as systems; no unit price published.
SXM | NVLink 1800 GB/s | CUDA
Which models fit, and how many cards
Planning estimates. 4-bit column: one user at 8K context. FP8 column: 10 users at once at 16K context.
Design a deployment on NVIDIA B300 (Blackwell Ultra) 270GB
Preloaded with this hardware. Add your ERP, documents and users.
Questions
What is the largest model that fits on one NVIDIA B300 (Blackwell Ultra) 270GB?
Of the models we track, MiniMax-M3 (428B total, 23B active) fits on a single 270 GB card in 4-bit with one user at 8K context. More users or longer context need memory for the KV cache, so leave headroom.
How fast is NVIDIA B300 (Blackwell Ultra) 270GB for LLM inference?
Generation speed is mostly set by memory bandwidth: 8000 GB/s here. Each token reads the active weights once, so a model with 8 GB of active weights tops out near 1000 tokens per second per stream before efficiency losses; batching many users raises total throughput.
Is NVIDIA B300 (Blackwell Ultra) 270GB a good choice for on-prem AI?
Blackwell Ultra in HGX/DGX B300 8-GPU systems (shipping, about 14 kW per DGX), with 13.5 PFLOPS dense FP4 per GPU and 2x B200 attention throughput. 270GB per GPU is derived from NVIDIA's 2.1 TB per 8 GPUs; BF16/FP8 are NVIDIA sparse totals halved; 1100W per Lenovo SR680a V4 certified listing. Sold only as systems; no unit price published.