What Is Embedding?
Also known as: Vector Embedding, Text Embedding
Definition
An embedding is a list of numbers, typically several hundred to a few thousand dimensions, that represents the meaning of a piece of text, image, or code so that semantically similar items produce vectors that sit close together.
Embedding Explained
An embedding model is a neural network trained so that inputs with similar meaning map to nearby points in a high-dimensional space. Training uses contrastive objectives: pairs known to be related are pulled together, unrelated pairs are pushed apart. After training, the geometry of that space encodes semantics. Cosine similarity between two vectors becomes a usable measure of how related two passages are, which is what makes semantic search possible without any shared keywords.
Dimensionality is a design tradeoff rather than a quality ranking. Common enterprise embedding models output between 384 and 1536 dimensions. Higher dimensions capture finer distinctions but cost more storage, more memory in the index, and more compute per comparison. Many recent models support dimension truncation, letting you keep a prefix of the vector at modest accuracy loss, which is a practical lever when index memory becomes the binding constraint.
Embeddings capture topical and semantic similarity, and they are genuinely poor at exact identifiers. The vectors for part 44-7192 and part 44-7193 will be nearly identical because the strings are nearly identical, even though the parts may be unrelated. Negation is another weak spot: "the fixture passed inspection" and "the fixture did not pass inspection" embed close together. Both limitations are why production retrieval pairs vector search with keyword matching rather than replacing it.
For on-prem deployment, embedding models are small and cheap relative to the generation model. Strong open-weight embedding models run comfortably in one or two gigabytes of VRAM and process thousands of chunks per minute on a single GPU, so the same server hosting your LLM can usually host embeddings too. The important constraint is consistency: the exact same model and version must embed both your corpus and every incoming query, or similarity scores become meaningless.
Why It Matters
- Enables semantic search that finds the right procedure even when the user's wording shares no keywords with the document.
- Powers clustering and deduplication across service notes, nonconformance reports, and supplier correspondence at a scale manual review cannot reach.
- Embedding dimension directly drives vector database storage and index memory, making it a real infrastructure decision rather than a detail.
- Weakness on exact identifiers and negation explains most confusing RAG retrieval failures and points straight at hybrid search as the fix.
In Practice
Mixing embedding models is a silent failure. A team reindexed part of a corpus with a newer model while leaving the rest on the old one. Queries still returned results and nothing errored, but relevance collapsed for half the corpus because vectors from two different models occupy incompatible spaces.
Frequently Asked Questions
What is the difference between an embedding and a token?
A token is a discrete chunk of text, roughly a word fragment, that a model reads as one unit. An embedding is a continuous numeric vector representing meaning. Every token has an internal vector inside the model, but when people say embedding in a retrieval context they usually mean one vector representing an entire passage, produced by a dedicated embedding model.
Can I use the same model for embeddings and generation?
Technically you can derive embeddings from a generative LLM's hidden states, but purpose-trained embedding models almost always retrieve better and cost a fraction of the compute. Standard practice is to run a small dedicated embedding model for indexing and search, and a separate larger generative model to compose the answer from the retrieved passages.
Related Terms
Vector Database
A vector database is a system that stores high-dimensional embedding vectors and retrieves the ones most similar to a query vector, enabling semantic search where results match meaning rather than exact keywords.
RAG (Retrieval-Augmented Generation)
RAG (Retrieval-Augmented Generation) is an architecture that searches your own document store for passages relevant to a question, inserts them into the LLM prompt, and instructs the model to answer only from that retrieved evidence.
Token
A token is the basic unit of text a language model processes, usually a word fragment rather than a whole word. In English, one token averages roughly four characters, so about 100 tokens covers 75 words.
Go Deeper
Manufacturing AI Readiness Assessment
Score your manufacturing operation's readiness for AI across data, systems, people, and governance, and get a prioritized roadmap for closing the gaps.
On-Prem LLM Deployment Architecture: Reference Guide
Reference architecture for on-prem LLM deployment: inference servers, GPU sizing, RAG pipelines, and security zones for regulated manufacturers.
Small Language Models vs LLMs for Enterprise Workloads
Small language models vs LLMs for enterprise workloads: accuracy, cost, GPU footprint, and when a 7B model beats a frontier model for ERP automation.
Working with Embedding in a live environment? Our engineers do this every day - and our AI agents automate most of it.