# On-Prem Document AI GPU Throughput Calculator

> Source: https://netray.co/tools/on-prem-document-ai-gpu-throughput-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Size GPUs for on-prem document extraction from daily pages, seconds per page, operating hours and utilization.

## Category

on-prem-ai

## Industry

Aerospace & Defense

## Audience

- IT architect
- AI engineer
- CIO

## Tool

**Kind:** calculator

**Inputs:**

#### Pages to process per day

**Key:** pages_per_day

**Type:** number

**Default:** 12000

**Min:** 0

**Step:** 500

**Unit:** pages

#### Processing time per page per GPU

**Key:** sec_per_page

**Type:** number

**Default:** 3

**Min:** 0.1

**Step:** 0.1

**Unit:** sec

**Help:** Measure end to end on your model and hardware, including pre- and post-processing.

#### Effective concurrent pages per GPU

**Key:** concurrency

**Type:** number

**Default:** 2

**Min:** 1

**Max:** 32

**Step:** 1

**Unit:** x

**Help:** Batching or concurrency gain. Keep it low unless you have measured it.

#### GPUs installed

**Key:** gpus

**Type:** slider

**Default:** 2

**Min:** 1

**Max:** 16

**Step:** 1

**Unit:** GPUs

#### Processing window per day

**Key:** run_hours

**Type:** number

**Default:** 16

**Min:** 1

**Max:** 24

**Step:** 1

**Unit:** h

#### Usable utilization

**Key:** utilization

**Type:** number

**Default:** 70

**Min:** 10

**Max:** 100

**Step:** 5

**Unit:** %

**Help:** Allow for queueing, retries and maintenance.

**Outputs:**

#### Pages per GPU per day

**Key:** pages_per_gpu

**Expr:** run_hours * 3600 / sec_per_page * concurrency * utilization / 100

**Format:** integer

#### Total daily capacity

**Key:** capacity

**Expr:** pages_per_gpu * gpus

**Format:** integer

**Primary:** true

#### Load versus capacity

**Key:** load

**Expr:** pages_per_day / capacity * 100

**Format:** percent

#### GPUs needed for the volume

**Key:** gpus_needed

**Expr:** ceil(pages_per_day / pages_per_gpu)

**Format:** integer

**Primary:** true

#### Spare pages per day

**Key:** headroom

**Expr:** capacity - pages_per_day

**Format:** integer

**Chart:**

#### Daily page capacity by GPU count

**Type:** bar

**X:** gpus

**From:** 1

**To:** 16

**Steps:** 16

**Series:**

- capacity

**Disclaimer:** Throughput depends on model, resolution, page density and serving stack. Benchmark your own documents before buying hardware.

## Content

**Intro:** Running document extraction inside your network means you own the capacity. This calculator turns a daily page volume into a GPU count so you can size an on-prem or air-gapped deployment before asking for budget.

**Sections:**

#### How to use the calculator

**Body:** Enter the pages you must process each day and the measured time to process one page on one GPU. Add the effective concurrency you actually achieve through batching, the hours in your processing window, and a utilization factor. The calculator returns capacity per GPU, total capacity, load as a share of capacity and the number of GPUs the volume needs.

**Bullets:**

- Benchmark with your real documents: scans, drawings and tables are slower than clean text.
- Keep concurrency conservative until measured.
- Plan for the monthly peak, not the average day.

#### The formula

**Body:** Pages per GPU per day equal the processing window in seconds divided by seconds per page, times effective concurrency, times utilization. Total capacity multiplies that by GPU count, and GPUs needed is daily pages divided by capacity per GPU, rounded up. The model treats work as evenly spread across the window, which real arrivals rarely are, so the headroom figure matters.

#### How to read the result

**Body:** Aim for load well under 100 percent so that spikes drain within the window. If you need an extra GPU only for a few peak days a month, consider a longer processing window, queueing low-priority batches overnight, or using a smaller specialized OCR model for simple documents and a larger vision-language model only for hard ones.

#### Common mistakes

**Body:** Quoting tokens per second instead of pages per second ignores that page length and resolution vary. Forgetting to size for redundancy leaves no capacity when a card fails. Forgetting memory is the other mistake: the model and its context must fit in GPU memory, which can limit concurrency before compute does.

**Cta:** Netray deploys open-weight document models on customer GPUs with vLLM or Ollama, and can run a benchmark on your own documents to give real seconds-per-page figures.

**Faq:**

#### How many pages per second can a GPU process?

**Answer:** It varies widely with the model, page resolution, page density and serving setup. Classic OCR engines are fast, vision-language models are slower. Measure on your own documents rather than relying on a general figure.

#### Do I need a GPU for OCR at all?

**Answer:** Not always. Classic OCR engines run on CPU for moderate volumes. GPUs matter when you use layout models or vision-language models for hard documents, or when volumes are high.

#### What utilization is realistic?

**Answer:** Plan for well below 100 percent to allow for queueing, retries and maintenance. Many teams size for 60 to 80 percent and treat the rest as headroom.

#### Should I size for peak or average?

**Answer:** Size for the peak you must meet, such as month-end, or lengthen the processing window and queue lower-priority work. Sizing for the average leaves delays on the days that matter.

## Search terms

on prem OCR GPU sizing, document AI GPU requirements, vision language model pages per day, air gapped document processing hardware, OCR throughput calculator, how many GPUs for document extraction, vLLM document throughput

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json