How to Fix Postgres pgvector Index Build Timing Out on Large Embeddings Dataset

Creating a vector index on a large table of embeddings takes far longer than expected, or times out entirely before completing. Building an index over millions of high-dimensional vectors is genuinely memory- and CPU-intensive work, and pgvector's default settings are often too conservative for datasets at real production scale.

The Problem

Creating an index on an embeddings column runs for a very long time and eventually fails or gets killed:

CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

ERROR: canceling statement due to statement timeout

Or, on a memory-constrained instance, it fails with an out-of-memory error instead:

ERROR: out of memory
DETAIL: Failed on request of size 268435456 in memory context "HNSW build context"

Why It Happens

Building a vector index — whether HNSW (Hierarchical Navigable Small World) or the older IVFFlat — requires PostgreSQL to hold a substantial working set in memory while constructing the index's internal graph or cluster structure, and this cost scales with both the number of vectors and their dimensionality. This differs meaningfully from building a typical B-tree index, which is comparatively lightweight. Common causes of the timeout or memory failure:

• maintenance_work_mem set too low for the dataset size — this is the specific memory setting PostgreSQL uses for index builds (separate from work_mem, which governs query execution), and its default is often far too conservative for a large embeddings table.
• A statement_timeout configured for normal query workloads being applied to what is fundamentally a long-running maintenance operation, cutting off an index build that was actually progressing normally, just slowly.
• High vector dimensionality (1536 dimensions for OpenAI embeddings, for example) multiplying the memory and compute cost per vector compared to lower-dimensional embeddings.
• HNSW's build parameters (m and ef_construction) set higher than necessary for your accuracy requirements, trading longer build times and more memory for index quality you may not actually need.
• Running the build on an undersized instance that simply doesn't have enough RAM or CPU to complete the operation in a reasonable time regardless of configuration tuning.

The Fix

Raise maintenance_work_mem for the session running the index build — this is the most impactful single setting for this specific operation:

SET maintenance_work_mem = '2GB';
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

If the build is being cut off by a statement timeout rather than genuinely failing on memory, raise or disable it specifically for this operation:

SET statement_timeout = 0; -- disable timeout for this session only
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

If pgvector was built with parallel build support (available in recent pgvector versions) and your PostgreSQL instance has multiple CPU cores available, enable parallel workers to significantly speed up the build:

SET max_parallel_maintenance_workers = 7;
SET maintenance_work_mem = '4GB';
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);

Consider whether the default m and ef_construction values are higher than your accuracy requirements actually need — lowering them reduces both memory usage and build time, at the cost of some recall accuracy during searches:

-- Lower values: faster build, less memory, slightly lower recall
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 8, ef_construction = 40);

For genuinely very large datasets where even a well-tuned build is impractical on your current instance, consider building the index on a larger, temporarily upsized instance (many managed platforms let you scale up compute for a maintenance window and scale back down afterward), since the index build cost is one-time while the ongoing query benefit is continuous.

Still Not Working?

If the build still fails even after tuning memory and timeout settings, run it as a background job outside of an interactive session entirely, so it isn't tied to a client connection timing out independently of PostgreSQL's own settings:

nohup psql -c "SET maintenance_work_mem='4GB'; CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);" > index_build.log 2>&1 &

Monitor its progress directly through PostgreSQL's own index-build progress view, available in recent PostgreSQL versions, which gives visibility into how far along a long-running build actually is rather than waiting blindly:

SELECT phase, blocks_done, blocks_total, tuples_done, tuples_total
FROM pg_stat_progress_create_index;

If the build is progressing steadily but is simply going to take a long time regardless of tuning, consider using CREATE INDEX CONCURRENTLY so the table remains fully available for reads and writes during the build, accepting a longer total build time in exchange for zero downtime on the table being indexed:

CREATE INDEX CONCURRENTLY ON items USING hnsw (embedding vector_cosine_ops);