When you add vector search to PostgreSQL with pgvector, one of the first decisions is which approximate nearest-neighbor index to use: IVFFlat or HNSW. Both avoid comparing a query embedding against every row, both can make semantic search dramatically faster, and they work in completely different ways.
IVFFlat clicked for me quickly, because it maps onto K-Means clustering. HNSW took much longer — not because vector search is hard, but because HNSW is a graph structure and I kept trying to understand it with the clustering model I had already built for IVFFlat.
That confusion turned expensive when a production search issue produced this:
hnsw.ef_search Approximate SQL query time
-------------- --------------------------
150 7 ms
300 21 ms
600 4,900 msDoubling ef_search from 300 to 600 did not double latency. It created a cliff.
What both indexes are solving
Suppose you have a million product embeddings and a user searches for beach wedding dress. Without an index, PostgreSQL does an exact search:
SELECT id, title
FROM products
ORDER BY embedding <=> $1
LIMIT 24;Accurate, but it may compute the distance to every eligible vector before sorting. Approximate indexes search only a carefully chosen slice instead, at the risk of occasionally missing a true nearest neighbor — so tuning them is a balance between recall and latency.
IVFFlat: partition the space
IVFFlat divides the vector space into groups called lists, which you can mentally treat as clusters.
CREATE INDEX products_embedding_ivfflat_idx
ON products
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 1000);A million vectors across 1,000 lists is roughly 1,000 vectors per list — unevenly distributed in practice, but close enough as a mental model. One list ends up holding running shoes and trainers, another wedding dresses and bridal gowns, another denim.
When a query arrives, PostgreSQL finds which lists sit closest to the query embedding, and ivfflat.probes decides how many to actually search. SET LOCAL ivfflat.probes = 10; is about 10,000 candidate vectors here. probes = 1 searches only the closest list — fast, lower recall. probes = 100 searches much more of the index — higher recall, slower. Once probes reaches the total list count the search is effectively exact, and PostgreSQL may decide the index isn't worth using.
lists is the build-side parameter. Too few and each list holds too many vectors, so queries scan too much. Too many and each list is tiny, so you need more probes to hold recall steady. pgvector suggests starting near rows / 1,000 up to a million rows and sqrt(rows) beyond that — heuristics, not rules.
Divide vectors into neighborhoods, then choose how many neighborhoods to open.
HNSW: navigate a graph
HNSW does not cluster anything. It builds a graph where each node is a stored vector and each edge is a connection to a nearby one. There is no "dress cluster" anywhere in it — only neighbor relationships, each node knowing a handful of other vectors that were close, or useful for navigation, when the index was built.
The "H" stands for Hierarchical, and the graph has several levels:
Top layer:
A ---------------------- M ---------------------- Z
Middle layer:
A ------- F ------- M ------- R ------- Z
Bottom layer:
A - B - C - D - E - F - G - H - I - J - K - L - M ...Upper layers have fewer nodes and longer-range connections; lower layers have more nodes and finer local detail. It works like navigating a city — motorways, then major roads, then local streets. To find a restaurant you take major roads to roughly the right area, then smaller roads to refine. HNSW does the same: enter at an upper layer, move toward nodes closer to the query, descend, search more broadly at the bottom, and return the closest results found.
What ef_search actually means
This is the parameter that causes the confusion. It's tempting to read hnsw.ef_search = 600 as start at one node, visit its nearest neighbor, and continue one-by-one until 600 nodes have been visited. That is not what happens. HNSW explores multiple promising routes at once, keeping two candidate collections — usually priority queues — one for nodes still worth exploring and one for the best results so far. It repeatedly takes a candidate, measures its neighbors against the query, keeps the useful ones, discards those no longer competitive, and stops when further exploration is unlikely to improve the results.
So ef_search = 600 means search the graph with a candidate working set of breadth 600. It does not mean exactly 600 vectors get compared, or exactly 600 graph hops occur, or that each of the 24 requested results receives 25 comparisons. Nor is there an equivalent of 600 × cluster size, because there are no clusters. A larger ef_search simply lets the algorithm hold and explore more competing candidates before committing — usually better recall, always more work. pgvector's default is 40.
This is also why LIMIT 24 and ef_search = 600 are unrelated numbers. LIMIT is how many rows come back; ef_search is how broadly the graph is explored before those rows are ranked and cut. Our application had tied them together:
ef_search = max(limit * 25, 500)An ordinary 24-result query became ef_search = 600; a 100-result query became 2500. The 25 was an application-level guess, not an HNSW rule. It tied search breadth to result count while ignoring the latency and recall characteristics of the actual production index.
The latency cliff
Benchmarked against a production database dump, that guess cost roughly 7 ms at ef_search = 150, 21 ms at 300, and 4,900 ms at 600. One plausible explanation is that the wider search crossed a threshold where substantially more index pages had to come from disk rather than cache. The lesson is not that 600 is universally bad:
HNSW latency is workload-specific and may not scale linearly with
ef_search.
Going from 300 to 600 does not turn 20 ms into 40 ms. It can just as easily turn it into several seconds, depending on graph size, cache availability, filtering, PostgreSQL configuration, and disk behavior.
We capped it at 300. SQL vector search dropped from roughly 9,005 ms to 25 ms, and the full API request from 28 seconds to 5.6. At that point the index was no longer the bottleneck — most of what remained was generating the query embedding through an external API. Which is a debugging principle in itself: don't optimize "vector search" as one block. Time embedding generation, retrieval, filtering, reranking, and generation separately.
The two build parameters
CREATE INDEX products_embedding_hnsw_idx
ON products
USING hnsw (embedding vector_cosine_ops)
WITH (
m = 16,
ef_construction = 64
);m caps how many connections each node gets per layer — lower means fewer edges, a smaller index, faster builds, and potentially lower recall. It's a limit, not a guarantee that every node has exactly m neighbors. ef_construction is how much effort HNSW spends finding good neighbors while building; higher means slower builds and better graph quality.
Don't confuse the two ef parameters: ef_construction applies when building the index, ef_search when querying it.
At a glance
Area IVFFlat HNSW
---- ------- ----
Structure Lists or partitions Hierarchical neighbor graph
Analogy Clustering Road network
Build speed Usually faster Usually slower
Memory usage Usually lower Usually higher
Query tuning probes ef_search
Build tuning lists m, ef_construction
Speed/recall trade-off Good, sensitive to tuning Often stronger
Create before loading data? Not recommended Yes
Changing data More operational care Generally convenient
Typical strength Lower memory, faster builds High recall at low latencyChoosing between them
You'll see rules like "IVFFlat below 30,000 vectors, HNSW above 50,000." Useful anecdotes, not thresholds — the real answer depends on dimensions, memory, storage speed, concurrency, acceptable recall, filters, and your latency target. The question isn't at what row count is HNSW better? but at what row count does my setup stop meeting my targets on my infrastructure?
For a few thousand vectors, exact search is often fast enough, and it gives you the baseline you need to measure approximate recall at all. IVFFlat suits constrained memory, real build-time limits, and stable datasets, if you're willing to tune lists and probes. HNSW suits substantial growth, tight latency with high recall, and data that changes often enough that retuning list counts becomes a chore.
Dimensions matter as much as row count: a vector uses four bytes per dimension, so 1,536 dimensions is roughly 6 GB per million embeddings before index links, WAL, and page overhead. pgvector's halfvec halves that at some cost to precision.
Migrating later is cheap — both index types can coexist on the same column, so you build the new one with CREATE INDEX CONCURRENTLY, run ANALYZE, confirm with EXPLAIN (ANALYZE, BUFFERS) that the planner uses it, then drop the old one. The SQL is the easy part; proving the new index behaves under realistic production queries is the real work.
Benchmark recall, don't guess it
To know whether approximate results are good, compare them against a forced exact scan:
SET LOCAL enable_indexscan = off;
SELECT id
FROM products
ORDER BY embedding <=> $1
LIMIT 24;Run the same query with SET LOCAL hnsw.ef_search = 100; and count the overlap: 22 of the exact top 24 is recall@24 = 91.7%. Repeat across representative queries — broad and specific, rare and common, with and without filters — never a single one like beach wedding dress. Then compare configurations:
Configuration P95 latency Recall@24
------------- ----------- ---------
ef_search=40 8 ms 86%
ef_search=80 12 ms 92%
ef_search=150 18 ms 96%
ef_search=300 30 ms 98%
ef_search=600 5,000 ms 99%Illustrative numbers, except the production latencies above. The best setting is not the highest-recall one: if 300 → 600 buys 98% → 99% recall for 30 ms → 5 seconds, it isn't worth it. Start from the defaults and walk ef_search up through 80, 150, 300, going higher only when benchmarks justify it.
Treat tuning formulas as starting points, not truths.
Filtering changes the picture
Most production vector searches carry filters — tenant, category, stock status. Approximate indexes may find vectors first and filter afterward, which creates a familiar failure:
HNSW finds 40 close candidates
↓
Most belong to another tenant or category
↓
Only 8 survive the filter
↓
LIMIT 24 cannot be satisfiedRecent pgvector versions support iterative index scans for both index types, which keep scanning when filtering strips out too many results. You should still index selective filter columns normally, consider partitioning for hard tenant boundaries, and benchmark filtered queries separately — a benchmark without production filters can be badly misleading.
The mental model
IVFFlat divides the vector space into lists and searches the nearest N of them: lists is how many neighborhoods exist, probes is how many get opened.
HNSW connects vectors to their neighbors in a layered graph and navigates it: m is graph connectivity, ef_construction is build-time search effort, ef_search is query-time search breadth.
And ef_search = 600 never meant "walk through 600 nodes in a straight line." It meant "explore the graph much more broadly, then return only the nearest 24" — a multiplier nobody had benchmarked, pushing an ordinary query across a latency threshold nobody had measured.
Vector index tuning should be driven by recall benchmarks, query latency, cache behavior, filtering, and production infrastructure — not by an arbitrary formula.
Choose the index on evidence, tune it with real queries, and measure every stage of the pipeline separately.
I write about the systems behind shipping AI features — RAG pipelines, evals, and the gap between demo and production — in my newsletter, AI Shipped. New issue every week.