On-Prem Document AI GPU Throughput Calculator
Running document extraction inside your network means you own the capacity. This calculator turns a daily page volume into a GPU count so you can size an on-prem or air-gapped deployment before asking for budget.
Your numbers
Measure end to end on your model and hardware, including pre- and post-processing.
Batching or concurrency gain. Keep it low unless you have measured it.
Allow for queueing, retries and maintenance.
Your results
Throughput depends on model, resolution, page density and serving stack. Benchmark your own documents before buying hardware.
Daily page capacity by GPU count
Sweeps GPUs installed from 1 to 16, holding your other inputs fixed.
On-prem benchmark
Optional: ask about benchmarking a document model on your own pages and hardware.
Optional. The tool and every export work without this. No spam.
How to use the calculator
Enter the pages you must process each day and the measured time to process one page on one GPU. Add the effective concurrency you actually achieve through batching, the hours in your processing window, and a utilization factor. The calculator returns capacity per GPU, total capacity, load as a share of capacity and the number of GPUs the volume needs.
- Benchmark with your real documents: scans, drawings and tables are slower than clean text.
- Keep concurrency conservative until measured.
- Plan for the monthly peak, not the average day.
The formula
Pages per GPU per day equal the processing window in seconds divided by seconds per page, times effective concurrency, times utilization. Total capacity multiplies that by GPU count, and GPUs needed is daily pages divided by capacity per GPU, rounded up. The model treats work as evenly spread across the window, which real arrivals rarely are, so the headroom figure matters.
How to read the result
Aim for load well under 100 percent so that spikes drain within the window. If you need an extra GPU only for a few peak days a month, consider a longer processing window, queueing low-priority batches overnight, or using a smaller specialized OCR model for simple documents and a larger vision-language model only for hard ones.
Common mistakes
Quoting tokens per second instead of pages per second ignores that page length and resolution vary. Forgetting to size for redundancy leaves no capacity when a card fails. Forgetting memory is the other mistake: the model and its context must fit in GPU memory, which can limit concurrency before compute does.
Frequently Asked Questions
How many pages per second can a GPU process?
It varies widely with the model, page resolution, page density and serving setup. Classic OCR engines are fast, vision-language models are slower. Measure on your own documents rather than relying on a general figure.
Do I need a GPU for OCR at all?
Not always. Classic OCR engines run on CPU for moderate volumes. GPUs matter when you use layout models or vision-language models for hard documents, or when volumes are high.
What utilization is realistic?
Plan for well below 100 percent to allow for queueing, retries and maintenance. Many teams size for 60 to 80 percent and treat the rest as headroom.
Should I size for peak or average?
Size for the peak you must meet, such as month-end, or lengthen the processing window and queue lower-priority work. Sizing for the average leaves delays on the days that matter.
Netray deploys open-weight document models on customer GPUs with vLLM or Ollama, and can run a benchmark on your own documents to give real seconds-per-page figures.
Related Tools
OCR Human Review Queue Staffing Calculator
Size the number of reviewers needed for a document capture exception queue, including peak days and the impact of the confidence threshold.
AI Agents & AutomationExtraction Precision, Recall and F1 Calculator
Score a document extraction or classification model with precision, recall and F1 from a labeled sample.
Costing & FinanceInvoice Processing Cost Per Invoice Calculator
Work out your fully loaded cost per invoice including labor, software, OCR page fees and exception handling outside AP.
Quality & InspectionOCR Field Accuracy to Document Accuracy Calculator
See why 98 percent field accuracy does not mean 98 percent clean documents, and what field accuracy a target requires.
More On-Prem AI tools
See all- On-Prem LLM Total Cost of Ownership Calculator
- GPU Sizing Calculator for LLM Inference
- On-Prem vs Cloud AI Cost Comparison Calculator
- Self-Hosted LLM Hardware Estimator
- Air-Gapped AI Readiness Assessment
- LLM API vs Self-Hosted Cost Calculator
- RAG Infrastructure Sizing Calculator
- Open-Weight Model Selector
- On-Prem AI ROI Calculator