# Why switching embedding models breaks your vector database, and why you can't just resize the column > Source: https://netray.co/troubleshoot/embedding-dimension-mismatch > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Different embedding models produce vectors of different fixed dimensions, so swapping models without re-embedding the entire corpus produces a hard dimension mismatch error or, worse, silently meaningless similarity scores if the column is resized without re-indexing. There is no shortcut: changing the embedding model requires re-embedding every document and rebuilding the vector index from scratch. ## Error Title Embedding dimension mismatch after switching models ## Error Signature ERROR: different vector dimensions 1536 and 768 ## Alternate Signatures - ValueError: shapes (768,) and (1536,) not aligned - vector must have at least 1 dimension error after model change - IndexError when comparing embeddings of different sizes ## Category rag-retrieval ## Stack - vector-db - huggingface ## Affects Any vector database with a fixed-dimension column, triggered specifically by upgrading, downgrading, or swapping the embedding model. ## Quick Fix ### Fastest path through an embedding model migration **Steps:** - Confirm the new model's output dimension with a quick test embedding call and compare it against your vector column's declared dimension. - Do not attempt to pad, truncate, or reinterpret vectors between different dimensions or models; treat this as a full migration, not a schema patch. - Create a new vector column or table sized for the new model's dimension. - Re-embed every document in the corpus with the new model and write into the new column or table. - Rebuild the HNSW or IVFFlat index on the new column, then cut over queries only after validating retrieval quality on the new embeddings. - Drop the old column or table only after the new one is validated in production. ## Symptoms - A database error explicitly reporting mismatched vector dimensions - Retrieval quality collapses immediately after upgrading or swapping the embedding model with no other code changes - Some rows in the vector table were embedded with the old model and others with the new one, producing inconsistent similarity scores - Application code throws a shape mismatch error when comparing query and document vectors - Everything runs without error but similarity scores are uniformly low and rankings look random ## Causes ### The vector column was declared for the old model's dimension and the new model outputs a different size **Likelihood:** most-common **Explanation:** pgvector and most vector databases enforce a fixed dimension per column or index at creation time. Every embedding model has a fixed, non-negotiable output size determined by its architecture, and there is no valid way to compare or store vectors of different dimensions in the same column. **Fix:** Create a new column or table sized for the new model's exact output dimension, and treat model swaps as a full re-embedding migration rather than an in-place update. **Commands:** - ALTER TABLE items ADD COLUMN embedding_new vector(1024); - SELECT vector_dims(embedding) FROM items LIMIT 1; ### Some rows were re-embedded with the new model while others still hold vectors from the old model **Likelihood:** common **Explanation:** If a migration is done incrementally in place without a full cutover, the table can end up with a mix of old-dimension and new-dimension vectors, or same-dimension vectors from two different models with incompatible geometry, silently corrupting similarity search for the entire table. **Fix:** Never mix embeddings from two different models or versions in the same column. Migrate to a new column or table, backfill it completely, validate it, and only then switch queries over and drop the old data. ### Application code assumes a fixed dimension constant that was not updated after the model change **Likelihood:** occasional **Explanation:** Many RAG codebases hardcode the embedding dimension in config or as a magic number used for validation, padding, or truncation logic. If that constant is not updated alongside the model swap, code paths that check or reshape vectors will throw shape errors even after the database schema is correctly updated. **Fix:** Search the codebase for the old dimension value as a literal and replace it with a single config-driven constant read from the embedding model's own reported output size, not hardcoded twice. **Commands:** - grep -rn '768' --include='*.py' . ### Truncated or Matryoshka-style embeddings were sliced to the wrong length **Likelihood:** rare **Explanation:** Some newer embedding models support Matryoshka representation learning, where a shorter prefix of the full vector is still a valid, lower-fidelity embedding. If code slices to an arbitrary length that does not match either the model's native full size or one of its officially supported truncation points, similarity scores become unreliable in ways that don't always throw an outright dimension error. **Fix:** Only truncate embeddings to lengths the model's documentation explicitly supports, and re-index consistently at that exact length across the entire corpus and all queries. ## Diagnostics ### Check the actual dimension of stored vectors **Command:** SELECT vector_dims(embedding) FROM items LIMIT 5; **Interpretation:** If this returns more than one distinct value across rows, the table has a mixed-dimension corruption problem that must be fully re-embedded and cannot be patched with a schema change alone. ### Check the declared column dimension **Command:** SELECT atttypmod FROM pg_attribute WHERE attrelid = 'items'::regclass AND attname = 'embedding'; **Interpretation:** Compare this against the output of a fresh test call to your embedding model; if they differ, either the schema needs to be migrated to the new dimension or the wrong model is being called. ### Test-embed a sample string with the current model and print its length **Command:** python -c "from your_embedder import embed; print(len(embed('test')))" **Interpretation:** This confirms exactly what dimension the currently configured model produces, which should be the single source of truth for both the schema and any hardcoded config values. ## Prevention - Store the embedding model name and dimension as metadata alongside the vector table, and validate it programmatically before any write. - Never hardcode the embedding dimension as a magic number in more than one place; derive it from the model at startup. - Treat any embedding model change as a full corpus migration project with a validation gate, not a config toggle. - Keep the old embedding table available, read-only, until the new one is validated in production, so rollback is possible. - Run a retrieval quality regression test immediately after any embedding model change before removing the old data. ## When To Escalate If you are re-embedding a very large corpus, or need to support multiple embedding models simultaneously for different tenants or use cases, this becomes an infrastructure and pipeline design question (parallel embedding throughput, storage cost, zero-downtime cutover) rather than a one-off migration script, and is worth planning explicitly before starting the re-embed. ## Faq ### Can I convert a 768-dimension embedding to 1536 dimensions with padding or interpolation? **Answer:** No. Padding, truncating, or interpolating vectors between models does not produce a meaningful embedding; the dimensions of two different models encode entirely different learned representations, so mathematical operations to force compatibility only produce vectors that look valid but carry no real semantic relationship to either model's actual embedding space. ### How long does re-embedding a large corpus take? **Answer:** It depends primarily on embedding model throughput and corpus size; a mid-sized embedding model on a single GPU can typically process several hundred to a few thousand chunks per second in batches, so a corpus of a few million chunks is usually a multi-hour to one-day job if batched properly. ### Do I need to rebuild the vector index after re-embedding, or just the data? **Answer:** Both. The HNSW or IVFFlat index is built directly from the vector data at creation time, so simply updating the underlying vectors in place without rebuilding the index leaves the index referencing stale or dimensionally invalid data. Always rebuild the index as the final migration step. ### Is it safe to run two embedding models side by side during migration? **Answer:** Yes, as long as they write to separate columns or tables and queries are explicitly routed to the matching one. The danger is only in mixing outputs from two models within a single column or index, which silently corrupts similarity search. ## Search terms embedding dimension mismatch, pgvector different vector dimensions error, re-embed corpus after model change, vector database schema migration, embedding model upgrade RAG, vector_dims pgvector, Matryoshka embeddings truncation --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json