# Chunking

> Source: https://netray.co/glossary/chunking
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Glossary

## Short answer

Chunking is the step in a RAG pipeline that splits long documents into smaller passages, usually a few hundred tokens each, so each can be embedded, indexed and retrieved on its own. Chunk boundaries strongly affect answer quality.

## Aliases

- document chunking
- text chunking
- text splitting

## Category

on-prem-ai

## Full Definition

A retrieval system cannot embed a 300-page maintenance manual as one vector and expect useful search. It splits the document into chunks, embeds each one, and stores them in a vector database. At question time it retrieves the few chunks most similar to the question and hands them to the LLM as context.

The simplest method is fixed-size chunking, for example 500 tokens with 50 to 100 tokens of overlap so a sentence cut at a boundary still appears whole in one chunk. It is fast but ignores structure. Structure-aware chunking splits on headings, numbered steps, table rows or page sections, and keeps a heading path such as "Section 7.3 Torque specifications" attached to every chunk as context.

Manufacturing documents punish naive splitting. A work instruction whose steps reference "the part above", a table of tolerances split across two chunks, or a revision block separated from its procedure all produce confident but wrong answers. Good pipelines keep tables whole, store document number and revision as metadata, and filter retrieval to the current effective revision.

There is no universal best size. Small chunks give precise matches but lose context. Large chunks keep context but dilute the embedding and waste the context window. Teams tune size against a set of real questions with known correct answers.

## Why It Matters

- Bad chunking is one of the most common reasons a RAG system returns incomplete or wrong answers.
- Attaching revision and document metadata prevents answers from superseded procedures.
- Chunk size drives cost and latency because retrieved text fills the context window.
- It is cheap to improve: re-chunking needs no model training.

## In Practice

A 120-page quality manual is split by heading into about 400 chunks of 300 to 600 tokens. Each chunk carries doc_id QM-001, revision F, and its heading path. A question about calibration intervals retrieves 4 chunks from section 7.1.5, all from revision F because older revisions are filtered out by metadata. The same manual chunked at fixed 1,000 tokens without overlap would have split the interval table in half.

## Faq

### What chunk size should I use?

**Answer:** A common starting point is 300 to 800 tokens with 10 to 15 percent overlap, then tune against real test questions. Procedures with numbered steps often work better split by step or section.

### Why include overlap?

**Answer:** Overlap keeps sentences that straddle a boundary intact in at least one chunk, which reduces cases where the answer is cut in half.

### How should tables be handled?

**Answer:** Keep a table in one chunk when it fits, repeat the header row if you must split it, and consider storing rows as structured data that a query tool can read instead.

### Does chunking change when documents are revised?

**Answer:** Yes. Re-index changed documents, store the revision as metadata, and retire or filter old chunks so answers cite only the effective revision.

## Search terms

chunking rag, document chunking strategy, chunk size overlap, rag best practices, chunking pdf manuals, structure aware chunking, rag work instructions, text splitter

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json