pgvector Extension Upgrade Quirks and Index Rebuilds

A field reference for upgrading pgvector safely: which version bumps add new types and operator classes, when an ivfflat or hnsw index must be rebuilt versus left in place, and how to cross a major PostgreSQL version without silently degrading recall.

Up: Compatibility Matrix Synchronization — this page is the pgvector-specific cell of that matrix, and it feeds the wider Extension Upgrade Planning & Compatibility Validation pipeline.

Context & When This Applies

pgvector adds a vector type plus approximate-nearest-neighbour index methods (ivfflat since 0.4, hnsw since 0.5) and, from 0.7, the companion types halfvec, bit-backed indexes, and sparsevec. It needs no shared_preload_libraries entry, so its upgrade is refreshingly database-local — but that simplicity hides a real trap: index operator classes and default distance handling can change between versions, and a stale index keeps answering queries without erroring while quietly returning worse results. This reference applies whenever you move pgvector between 0.5, 0.6, 0.7, and later on PostgreSQL 12–17, and it is the prerequisite for any workload where recall quality is a correctness property, not a nicety.

Because the visible failure mode is degraded results rather than an error, pgvector upgrades demand explicit post-upgrade verification of recall, not just a version check. Rehearse the bump on a production-sized clone through test environment routing and compare query results before and after, so a recall regression is caught in staging instead of by a user.

Deciding whether a pgvector upgrade needs an index rebuild A pgvector upgrade enters a decision. If the release only adds functions or types without changing existing operator classes, no rebuild is needed and the indexes keep working. If the release changes an operator class, the storage format, or the default distance handling for an index method already in use, every affected ivfflat and hnsw index must be rebuilt with REINDEX INDEX CONCURRENTLY, and recall must be re-measured. Crossing a PostgreSQL major with pg_upgrade preserves index files, so the same operator-class check decides whether a post-upgrade reindex is required. pgvector UPDATE new version applied opclass or format change? Yes → rebuild REINDEX CONCURRENTLY re-measure recall No → keep index functions/types added no rebuild required yes no

Concept: Operator Classes Decide Rebuild vs No-Rebuild

An ivfflat or hnsw index is built against a specific operator class — vector_l2_ops, vector_cosine_ops, vector_ip_ops, and from 0.7 their halfvec_* counterparts. The index’s on-disk structure encodes assumptions from the version that built it: hnsw graph layout, ivfflat centroid lists, and the distance function bound to each opclass. A pgvector upgrade that only adds new opclasses or types leaves existing indexes valid — they still reference the same opclass with the same semantics. An upgrade that changes an existing opclass’s storage or distance handling makes the old index structurally stale: it will still return rows, but the graph or lists no longer match the query-time distance computation, so recall drops. This is precisely the class of silent, call-time divergence the error categorization framework cannot catch, because nothing raises — which is why the pipeline treats a pgvector opclass change as a mandatory reindex, not an optional one.

The second concept is that pg_upgrade preserves index files byte for byte. Crossing a PostgreSQL major does not rebuild any index, so the same opclass-change question decides whether you must REINDEX afterward. Deciding between the in-place and logical routes is the subject of pg_upgrade vs a logical replication upgrade path; either way, the reindex decision is yours to make explicitly.

Runnable Implementation

The procedure upgrades pgvector, enumerates its ANN indexes, and rebuilds them concurrently only when a rebuild is warranted. REINDEX ... CONCURRENTLY avoids an AccessExclusiveLock so reads and writes continue during the rebuild.

-- 1. Transactional catalog upgrade. Adds any new types/opclasses; existing
--    indexes are untouched by this statement.
ALTER EXTENSION vector UPDATE;

-- 2. Enumerate every ivfflat/hnsw index and the opclass it was built against,
--    so the rebuild decision is data-driven rather than guessed.
SELECT c.relname            AS index_name,
       i.relname            AS table_name,
       am.amname            AS method,          -- ivfflat | hnsw
       oc.opcname           AS opclass
FROM   pg_index x
JOIN   pg_class c  ON c.oid = x.indexrelid
JOIN   pg_class i  ON i.oid = x.indrelid
JOIN   pg_am    am ON am.oid = c.relam
JOIN   pg_opclass oc ON oc.oid = ANY (x.indclass::oid[])
WHERE  am.amname IN ('ivfflat', 'hnsw')
ORDER  BY table_name, index_name;

-- 3. Rebuild WITHOUT blocking writes, per index flagged by the release notes.
REINDEX INDEX CONCURRENTLY idx_docs_embedding_hnsw;

Drive the enumerate-then-rebuild loop from Python with psycopg2 or asyncpg for extension automation scripts, and size each concurrent rebuild’s resource cost with threshold tuning for downtime windows — an hnsw rebuild on a large table is CPU- and memory-heavy even without an exclusive lock.

Expected Output & Verification

Step 1 advances the version; step 2 lists the indexes to consider:

    index_name          | table_name | method |    opclass
------------------------+------------+--------+------------------
 idx_docs_embedding_hnsw| docs       | hnsw   | vector_cosine_ops

The decisive verification is recall, not a version string. Run a fixed probe set before and after and compare the returned neighbours:

-- Sample recall: does the ANN index return the same top-k as an exact scan?
WITH q AS (SELECT embedding FROM docs WHERE id = 42)
SELECT d.id
FROM   docs d, q
ORDER  BY d.embedding <=> q.embedding      -- cosine distance
LIMIT  10;

If the post-upgrade top-k diverges materially from the pre-upgrade result (or from an exact SET enable_indexscan = off baseline), the index is stale and must be reindexed. Record the shipped pgvector version and the per-index opclass in compatibility matrix synchronization.

Edge Cases & Gotchas

A stale index degrades recall without ever erroring. This is the defining pgvector gotcha: after an opclass-changing upgrade the index still answers, just worse. There is no ERROR to categorize — the only signal is a recall regression against a known-good baseline. Always keep a golden query set from before the upgrade.

REINDEX CONCURRENTLY can leave an invalid index on failure. If a concurrent rebuild is interrupted, you are left with an INVALID index that the planner ignores:

WARNING:  cannot drop index idx_docs_embedding_hnsw_ccnew concurrently ...

Detect it and clean up before assuming the rebuild succeeded:

SELECT c.relname
FROM   pg_index x JOIN pg_class c ON c.oid = x.indexrelid
WHERE  NOT x.indisvalid;

Drop the invalid leftover and reissue the reindex; keep a snapshot and point-in-time recovery checkpoint so an interrupted rebuild has a clean restore target.

hnsw build memory can exceed maintenance_work_mem and spill. Large hnsw rebuilds that do not fit in maintenance_work_mem fall back to a much slower on-disk build. Raise maintenance_work_mem for the reindexing session and account for the extra memory in your window estimate rather than discovering the slowdown mid-rebuild.

Downgrading pgvector is not supported once a new type is in use. If any column uses halfvec or sparsevec introduced in a newer version, ALTER EXTENSION vector UPDATE TO '<older>' fails because the older version has no such type. Plan pgvector bumps as forward-only and rely on snapshot and point-in-time recovery for rollback rather than a version downgrade.