Vector Databases 101: What They Are, How They Work and When to Use One
A plain-English guide to vector databases: what they are, how embeddings and ANN search work, top use cases, and how to pick between pgvector and Milvus.
TL;DR
- A vector database stores data as numeric vectors (embeddings) and finds items by meaning, not by exact keyword match.
- It powers semantic search, retrieval-augmented generation (RAG), recommendations, image search, and deduplication.
- It works by converting text or images into embeddings, then using approximate nearest neighbour (ANN) search over an index like HNSW to return the closest matches fast.
- Popular options: Pinecone, Milvus, Weaviate, Qdrant, Chroma, and pgvector (vectors inside PostgreSQL).
- For most teams already on PostgreSQL, pgvector is the pragmatic start. Reach for Milvus or a managed service when you cross tens of millions of vectors or need heavy filtered search at scale.
What is a vector database?
A vector database is a database built to store and search high-dimensional vectors called embeddings, which are numeric representations of meaning. Instead of matching exact keywords, it finds the items whose vectors are closest to a query vector, so a search for 'car' also surfaces 'automobile' and 'vehicle'. It is the storage and retrieval layer behind semantic search and most retrieval-augmented generation (RAG) systems.
The short answer
A vector database finds things by meaning. You turn your text, images, or audio into embeddings, which are lists of numbers that capture meaning, store them, and then ask the database for the items closest to a new query. Because closeness in that number space maps to closeness in meaning, a search for 'how do I reset my password' returns the doc titled 'account recovery steps' even though they share no keywords.
That one capability is what makes modern AI search and retrieval-augmented generation work. If you are building a chatbot over your own docs, a semantic search box, or a recommendation engine, a vector database is almost certainly the retrieval layer underneath. The rest of this guide explains embeddings, how the search actually runs, when you need a dedicated vector database, and how to pick one.
What are embeddings?
An embedding is a list of numbers that represents the meaning of a piece of content. A model reads your text and outputs a vector, for example 1,536 numbers for OpenAI's text-embedding-3-small or 3,072 for text-embedding-3-large. Similar meanings produce vectors that sit close together; unrelated meanings sit far apart.
The useful part is that maths now stands in for meaning. 'King' minus 'man' plus 'woman' lands near 'queen'. 'Invoice overdue' lands near 'unpaid bill' and far from 'team lunch'. You generate embeddings with an embedding model (OpenAI, Cohere, Voyage AI, or an open-source sentence-transformers model), and the vector database stores and searches them. Picking the embedding model matters as much as picking the database, which we cover in choosing the right AI model for your product.
How do vector databases work?
Three moving parts: you embed your data, you store it in an index, and you search that index for nearest neighbours. Here is each step.
- Embed and store. Every document, chunk, or image is run through an embedding model once and stored as a vector, usually alongside metadata like source, date, or author. Do this for your whole corpus up front, then keep it current as new content arrives.
- Approximate nearest neighbour (ANN) search. At query time the database embeds the query and looks for the stored vectors closest to it. Checking every vector one by one (exact nearest neighbour) is accurate but too slow at scale. So vector databases use approximate nearest neighbour, or ANN, which trades a tiny amount of accuracy for a massive speed gain, returning results in milliseconds across millions of vectors.
- The index: HNSW, IVF and friends. ANN speed comes from the index, the data structure that organises vectors so the search can skip most of them. The most common is HNSW (Hierarchical Navigable Small World), a graph index that navigates from rough neighbourhoods to precise ones in a few hops; it is the default in Qdrant, Weaviate, Milvus, and pgvector. IVF (inverted file) clusters vectors and searches only the nearest clusters. FAISS, the library from Meta, implements several of these and underpins many systems. You tune the index to trade recall against speed and memory.
- Similarity metrics. 'Closest' needs a definition. Cosine similarity measures the angle between two vectors and is the common default for text. Dot product factors in magnitude too. Euclidean (L2) distance measures straight-line distance. The right metric depends on your embedding model, and most databases let you choose per collection.
What are vector databases used for?
Anywhere you need search by meaning rather than exact match. The main use cases:
| Use case | What the vector database does |
| Retrieval-augmented generation (RAG) | Retrieves the most relevant chunks of your data to feed an LLM as grounded context |
| Semantic search | Returns results by meaning, so synonyms and paraphrases match |
| Recommendations | Finds items similar to what a user liked, by content or behaviour embedding |
| Image and audio search | Finds visually or sonically similar media via multimodal embeddings |
| Deduplication and clustering | Groups near-identical records and flags duplicates |
| Anomaly detection | Flags vectors that sit far from every known cluster |
RAG is the one most teams reach for first. If you want a chatbot that answers from your own knowledge base, the vector database holds your content and hands the right passages to the model at query time. We walk through a full build in implementing RAG on a company wiki, and how the retrieval layer fits a broader stack in integrating LLMs into production applications.
Do you always need a dedicated vector database?
No, and this is where teams overspend. If you already run PostgreSQL, the pgvector extension adds vector columns and ANN search to the database you already operate, no new system to run. For a corpus in the tens or low hundreds of thousands of vectors, that is often all you need. Reach for a dedicated vector database (Milvus, Qdrant, Weaviate) or a managed service (Pinecone) when you cross into many millions of vectors, need heavy metadata filtering at speed, or want horizontal scaling and sharding out of the box.
Which vector database should you use?
The honest landscape, for teams choosing in 2026:
| Option | Best for | Watch out for |
| pgvector | Teams already on PostgreSQL; small to mid corpora | Very large scale needs careful indexing and tuning |
| Pinecone | Fully managed, fast to ship, no ops | Managed pricing; less control over internals |
| Milvus | Large scale, billions of vectors, self-hosted | More infrastructure to run and operate |
| Qdrant | Strong filtered search, Rust performance | Smaller ecosystem than the incumbents |
| Weaviate | Built-in hybrid search and modules | Opinionated model; learn its schema |
| Chroma | Prototyping and local development | Not aimed at large production scale |
pgvector vs Milvus: the practical call
This is the comparison most of our clients actually face: start simple inside Postgres, or stand up a dedicated engine. The straight answer:
| Factor | pgvector | Milvus |
| Scale sweet spot | Up to low tens of millions of vectors | Tens of millions to billions |
| Operations | None extra; it is your Postgres | A distributed system to run and monitor |
| Filtered search | Good, via SQL WHERE on metadata | Purpose-built for high-volume filtered ANN |
| Best when | You value one system and SQL joins | Vectors are the core workload at scale |
Our default recommendation: start with pgvector if you are already on PostgreSQL. It keeps your stack to one database, lets you join vectors against your relational data in plain SQL, and handles the majority of production RAG workloads we build. Migrate to Milvus (or a managed Pinecone) when you genuinely outgrow it, when vector search is the primary workload at tens of millions of records with demanding latency and filtering. Choosing the engine before you know your scale is how teams end up running infrastructure they do not need.
How to add a vector database to your product
A first vector-search feature is a short, well-trodden build:
- Chunk and clean your data. Split documents into passages of a few hundred tokens with slight overlap, and strip boilerplate. Chunking quality drives retrieval quality more than the database choice does.
- Choose an embedding model. Match the model to your content and budget. A general text-embedding model covers most cases; specialised or multilingual models earn their place for specific domains.
- Embed and load. Generate embeddings for every chunk and load them with metadata into your database of choice. Store the source so you can cite it later.
- Query, filter, and rank. Embed the user query, run ANN search, filter on metadata, and optionally re-rank the top results with a cross-encoder for precision.
- Ground the model and evaluate. Feed the retrieved passages to the LLM as context and measure retrieval quality with a real evaluation set. Retrieval you never measure is retrieval you cannot trust, which matters most in agentic systems (building agentic AI).
For real-world case studies, see how we engineered Getlem AI context and compliance platform and built the Spellbook AI legal assistant. Explore our full range of AI development services and machine learning development.
Building AI search, RAG, or agents?
Parallel Loop designs the retrieval layer to fit your data and budget, from pgvector to a managed vector store, so it is fast, accurate and not over-engineered. Book a free scoping call.
Parallel Loop pricing (USD): AI Agent Development from $10,000. MVP plus AI feature from $10,000. Custom enterprise AI builds quoted on scope.
Frequently Asked Questions
What is a vector database?
A database built to store and search embeddings, the numeric vectors that represent meaning. It finds items whose vectors are closest to a query vector, so it searches by meaning rather than exact keywords. It is the retrieval layer behind semantic search and most RAG systems.
How do vector databases work?
They embed your content into vectors, store those vectors in an index such as HNSW, and at query time run approximate nearest neighbour (ANN) search to return the closest vectors in milliseconds, ranked by a similarity metric like cosine similarity.
What are vector databases used for?
Retrieval-augmented generation (RAG), semantic search, recommendations, image and audio similarity search, deduplication, clustering, and anomaly detection, anywhere search by meaning beats exact keyword match.
What is the difference between pgvector and Milvus?
pgvector adds vector search to PostgreSQL, so you keep one database and query vectors with SQL; it suits small to mid corpora. Milvus is a dedicated, distributed vector database built for tens of millions to billions of vectors and high-volume filtered search at scale.
Do I need a vector database for RAG?
Almost always, yes. RAG retrieves the most relevant chunks of your data to feed an LLM, and a vector database is what makes that retrieval fast and meaning-based. For small corpora, pgvector inside your existing Postgres is often enough.
What are embeddings?
Embeddings are lists of numbers that represent the meaning of text, images, or audio. Similar meanings produce vectors that sit close together. An embedding model generates them, for example OpenAI text-embedding-3-small outputs 1,536-dimensional vectors.