Vector Databases

You have a million document chunks, each represented as a 1536-dimensional embedding vector. A user asks a question, you embed it, and you need the 10 most similar vectors. Exact nearest-neighbor search means computing the distance to every single vector — O(n) per query. At a million vectors and a few hundred queries per second, that's not going to work. Vector databases exist to solve this problem.

Approximate Nearest Neighbors (ANN)

The fundamental trade-off: ANN algorithms sacrifice a small amount of recall (they might miss the absolute best match occasionally) for orders-of-magnitude speed improvement. Instead of scanning every vector, they build index structures that narrow the search space dramatically.

A well-tuned ANN index can search a billion vectors in milliseconds while returning the correct top-10 results 95-99% of the time. For RAG and semantic search, that recall level is more than sufficient — you're retrieving context, not performing exact lookups.

HNSW: The Dominant Index

Hierarchical Navigable Small World (HNSW) graphs are the most widely used ANN index today. The idea builds on two concepts: small-world graphs (where most nodes can be reached through a small number of hops) and hierarchical layers.

HNSW constructs a multi-layer graph. The top layer is sparse — just a few well-connected nodes acting as entry points. Each lower layer adds more nodes with more local connections. Search works by greedy descent: start at the top layer, find the nearest node, drop to the next layer, find a closer node, repeat until you reach the bottom layer where the fine-grained search happens.

Key tuning parameters:

• M — the number of connections per node. Higher M = better recall, more memory, slower insertion
• ef_construction — controls index build quality. Higher values produce a better graph at the cost of slower indexing
• ef_search — controls query-time quality. Higher values check more candidates, improving recall but increasing latency

# Typical starting point for pgvector HNSW
CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 200);

# At query time
SET hnsw.ef_search = 100;

Alternative Index Types

IVF (Inverted File Index) — partitions vectors into clusters using k-means. At query time, only the nearest nprobe clusters are searched. Faster to build than HNSW, but recall drops if you set nprobe too low. Tuning knob: nprobe (more clusters searched = higher recall, more latency).

Product Quantization (PQ) — compresses vectors by splitting them into sub-vectors and quantizing each. Drastically reduces memory but introduces approximation error. Often combined with IVF (IVF-PQ) for large-scale systems where memory is the bottleneck.

Flat (brute force) — no index at all, exact search. Actually the right choice when you have fewer than ~50k vectors. No tuning, perfect recall, and fast enough at small scale.

Distance Metrics

Your choice of distance metric must match what your embedding model was trained with. Get this wrong and your search results will be meaningless.

• Cosine similarity — measures angle between vectors, ignoring magnitude. The most common choice for text embeddings. OpenAI and most embedding models use this
• Dot product (inner product) — equivalent to cosine when vectors are normalized. Some models (like those producing normalized embeddings) work with either
• Euclidean (L2) — measures straight-line distance. Less common for text, sometimes used for image embeddings

When in doubt, check your embedding model's documentation. If it says "cosine similarity," use cosine distance in your vector database.

Metadata Filtering

Real applications need filtered search: "find similar documents, but only from this tenant" or "only documents created after 2024." This is where the pre-filter vs post-filter problem emerges.

Post-filtering runs the ANN search first, then filters results. If the filter is selective (eliminates 99% of results), you might get back far fewer than the requested k results — or none at all.

Pre-filtering applies the filter before vector search. More accurate results, but it means the vector index needs to interact with metadata indexes, which is architecturally complex and can be slower.

Most mature vector databases now support pre-filtering, but performance varies significantly. Test with your actual filter selectivity.

Hybrid Search

Pure vector search finds semantically similar content but can miss exact keyword matches. Pure BM25 (keyword) search catches exact terms but misses paraphrases. Hybrid search combines both, typically using Reciprocal Rank Fusion (RRF) to merge the two ranked lists.

# Conceptual hybrid search
vector_results = vector_db.search(query_embedding, k=20)
keyword_results = bm25_index.search(query_text, k=20)
final_results = reciprocal_rank_fusion(vector_results, keyword_results)

This matters for RAG systems where users might search for specific product names, error codes, or technical terms that embeddings don't capture well.

The Options Landscape

pgvector — a PostgreSQL extension. If you already run Postgres, this is the path of least resistance. HNSW and IVF indexes, good enough for most applications up to a few million vectors. You keep your existing infrastructure, backups, and operational knowledge.

Pinecone — fully managed, serverless option. Simple API, handles scaling and indexing. Good for teams that don't want to operate infrastructure. Cost scales with usage.

Weaviate, Qdrant, Milvus — purpose-built vector databases with richer features (built-in hybrid search, multi-tenancy, advanced filtering). Worth evaluating when you outgrow pgvector or need features it doesn't support.

The Honest Take

Most applications don't need a dedicated vector database. If you're building a RAG system with fewer than a few million vectors, pgvector in your existing Postgres instance is almost certainly sufficient. It's one fewer service to deploy, monitor, and pay for.

Reach for a dedicated vector database when you hit a specific pain point: you need billion-scale search, you need advanced multi-tenancy, or pgvector's query latency doesn't meet your SLA. Let the problem justify the infrastructure, not the other way around.

Key Takeaways

• ANN indexes trade a small amount of recall for massive speed improvements — exact search is O(n) and doesn't scale
• HNSW is the dominant index type: a multi-layer graph searched by greedy descent, tuned via M, ef_construction, and ef_search
• Your distance metric must match your embedding model's training — cosine similarity is the most common for text
• Metadata filtering requires pre-filtering for accurate results; post-filtering can return incomplete result sets
• Hybrid search (vector + BM25) handles both semantic similarity and exact keyword matching
• pgvector in Postgres is sufficient for most applications — adopt a dedicated vector database when you hit a concrete scaling limit

Next, we'll get practical with LLM APIs — the mechanics of calling OpenAI, Anthropic, and Gemini, including streaming, tool calling, and token accounting.