# RAG Infrastructure Sizing Calculator: Vectors, RAM, and GPUs

> Source: https://netray.co/tools/rag-infrastructure-sizing-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Estimate vector storage, node RAM, generation GPUs, and monthly infrastructure cost for a retrieval-augmented generation deployment over your document corpus.

## Category

on-prem-ai

## Tool

**Kind:** calculator

**Inputs:**

#### Documents in corpus

**Key:** document_count

**Type:** number

**Default:** 250000

**Min:** 1000

**Max:** 10000000

**Step:** 1000

**Unit:** docs

**Help:** Total documents to index: manuals, drawings metadata, SOPs, contracts, tickets.

#### Average pages per document

**Key:** avg_pages_per_doc

**Type:** number

**Default:** 8

**Min:** 1

**Max:** 200

**Step:** 1

**Unit:** pages

**Help:** A page is roughly 500 words; long manuals raise this quickly.

#### Queries per day

**Key:** queries_per_day

**Type:** slider

**Default:** 2000

**Min:** 50

**Max:** 100000

**Step:** 50

**Unit:** queries

**Help:** RAG questions asked across all users and automations daily.

#### Embedding dimensions

**Key:** embedding_dim

**Type:** select

**Default:** 1024

**Help:** Vector width of your embedding model; higher is more accurate but heavier.

**Options:**

##### 768 (compact models)

**Value:** 768

##### 1024 (standard enterprise)

**Value:** 1024

##### 1536 (high-accuracy models)

**Value:** 1536

#### Deployment topology

**Key:** replication

**Type:** select

**Default:** 2

**Help:** Production deployments should run at least an HA pair.

**Options:**

##### Single node (dev/pilot)

**Value:** 1

##### HA pair (production)

**Value:** 2

**Outputs:**

#### Total vector chunks

**Key:** total_chunks

**Expr:** document_count * avg_pages_per_doc * 2

**Format:** number

**Description:** Assumes ~2 chunks per page at 400-500 tokens per chunk.

#### Vector index size

**Key:** vector_storage_gb

**Expr:** round(total_chunks * embedding_dim * 4 * 1.5 / 1073741824 * 10) / 10

**Format:** number

**Description:** Float32 vectors plus 50% HNSW index and metadata overhead, in GB per node.

**Primary:** true

#### RAM per vector node

**Key:** ram_per_node_gb

**Expr:** round(max(32, vector_storage_gb * 1.25 + 16))

**Format:** number

**Description:** Index held in memory plus OS and query headroom, in GB (32 GB floor).

#### Generation GPUs needed

**Key:** llm_gpus

**Expr:** ceil(queries_per_day * 800 * 3 / (28800 * 600))

**Format:** number

**Description:** GPUs for answer generation at ~800 output tokens per query with 3x peak factor.

#### Estimated monthly infrastructure cost

**Key:** est_monthly_cost

**Expr:** round(replication * (ram_per_node_gb * 6 + 350) + llm_gpus * 1400)

**Format:** currency

**Description:** Vector nodes priced by RAM plus amortized GPU serving cost.

**Primary:** true

**Disclaimer:** Estimates only. Chunking strategy, quantized vectors, and disk-backed indexes can change these numbers significantly. Validate with a pilot index over a corpus sample.

## Content

**Intro:** This calculator sizes the infrastructure behind an enterprise retrieval-augmented generation (RAG) deployment - vector index storage, node RAM, generation GPU count, and a monthly cost estimate - for teams planning to put an LLM over their document corpus. RAG sizing has two halves that scale independently: the retrieval side grows with corpus size, while the generation side grows with query volume. Enter your document counts, embedding model, and daily query load, and the tool produces a node-level specification you can take into architecture review.

**Sections:**

#### The sizing methodology

**Body:** Documents become chunks: at roughly 500 words per page and 400-500 token chunks with overlap, each page yields about two vectors. Each vector consumes embedding dimensions times four bytes as float32, and production HNSW indexes plus metadata add roughly 50% on top - so 250,000 eight-page documents at 1,024 dimensions produce about 4 million chunks and a 23 GB index. Because HNSW performs best fully in memory, node RAM is sized at 125% of the index plus a 16 GB base, with a 32 GB floor. Generation GPUs are sized from daily queries at about 800 output tokens each, a 3x peak-hour factor, and 600 tokens per second per GPU - which is why most mid-size corpora need exactly one generation GPU.

#### Benchmarks the defaults are built on

**Body:** The defaults model a realistic mid-market manufacturing knowledge base: engineering specs, quality procedures, maintenance manuals, and contracts. Numbers worth knowing when you sanity-check your own inputs:

**Bullets:**

- Chunking yield: ~2 chunks per page at 400-500 tokens with 10-15% overlap
- Float32 vector cost: 4 KB per chunk at 1,024 dims before index overhead
- HNSW overhead: 40-60% above raw vector bytes for links and metadata
- A 4M-chunk index serves sub-100ms retrieval comfortably on a 64 GB node

#### How to interpret and refine the result

**Body:** If the vector index lands under 50 GB, a simple HA pair of general-purpose nodes handles retrieval and your cost center is the generation GPU - which is the common case and good news, since GPU capacity is reusable across use cases. If the index exceeds a few hundred gigabytes, before buying bigger nodes consider scalar or binary quantization (2-4x smaller with minor recall loss), disk-backed indexes like DiskANN, or trimming the corpus - most enterprises find 20-40% of indexed documents are obsolete duplicates. Also stress-test query volume honestly: agentic workflows that fire multiple retrievals per user question can multiply effective load 3-10x.

#### How Netray builds production RAG on these numbers

**Body:** Netray designs and deploys on-prem RAG systems for manufacturers - typically over engineering documents, quality records, and ERP data from SyteLine or LN environments. We validate this sizing with a pilot index over a corpus sample, choose the embedding model and chunking strategy against your actual retrieval accuracy targets, and deliver the full stack: ingestion pipelines, vector database, reranking, generation serving, and evaluation harness. Air-gapped variants for defense programs are a specialty.

**Cta:** Size your RAG stack, then have Netray validate retrieval quality with a pilot index over a sample of your real corpus.

**Faq:**

#### Why does the calculator assume vectors live in RAM?

**Answer:** Because HNSW, the dominant production index type, degrades sharply when its graph pages to disk - latency jumps from milliseconds to seconds under load. For corpora up to a few tens of millions of chunks, RAM-resident indexes are the reliable default and RAM is cheap relative to the GPU side. At larger scales, disk-backed designs like DiskANN or quantized indexes change the economics, which is exactly the kind of threshold decision worth an architecture review.

#### Does query volume really only need one GPU in most cases?

**Answer:** Usually, yes - and it surprises people. Two thousand queries a day at 800 output tokens averages under one token per second of sustained generation; even with a 3x peak factor, a single modern GPU serving a quantized 14B-32B model has ample headroom. Query volume starts demanding multiple GPUs around 20,000-50,000 daily queries, or much earlier if agentic pipelines chain several LLM calls per user question.

#### What drives RAG answer quality more: the corpus size or the model?

**Answer:** Neither - retrieval quality does. In Netray evaluations, chunking strategy, embedding model choice, metadata filtering, and reranking explain most of the gap between disappointing and excellent RAG systems, while swapping a 14B for a 70B generation model moves scores far less. That is why we size infrastructure with modest generation GPUs and invest the savings in ingestion quality, hybrid search, and a proper evaluation harness with a gold question set.

## Search terms

rag infrastructure sizing, vector database sizing calculator, rag hardware requirements, enterprise rag architecture cost, embedding storage calculator, on-prem rag deployment, vector index memory requirements

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json