Keeping a Vector Index in Sync With a Moving Codebase

Retrieval tutorials index a folder of documents, ask three questions, and stop. Every hard problem in a production retrieval system lives in what happens next: the corpus changes, continuously, and an index built once starts lying almost immediately.

For a code assistant this is acute. A repository with a few active developers changes many times a day. An index built last week describes a codebase that no longer exists, and it will answer questions about it with complete confidence.

Re-indexing everything is not a strategy

The first answer is always a nightly job that re-embeds the corpus. It works at small scale and stops working for three reasons.

Cost. Embedding is priced per token. Re-embedding a large repository every night means paying for the ninety-nine percent that did not change, every night, forever.

Latency of correctness. A nightly rebuild means the index is wrong for up to a day. Someone asks about the function they merged this morning and gets the version it replaced.

Write amplification. Replacing every vector rewrites the whole index, which for HNSW means rebuilding a graph that took a long time to construct — while queries are being served against it.

Deletions are the case that actually hurts

Incremental updates get discussed as though additions and modifications were the whole problem. They are the easy half. If a new function is missing from the index, retrieval returns nothing for it and the user sees an unhelpful answer — bad, but visibly bad.

A deleted chunk that remains in the index is worse in kind. It is retrieved, it looks authoritative, it is cited with a file and line number, and it describes code that was removed three weeks ago. Nobody can tell from the answer that it is wrong. The system is not failing to help; it is fabricating with citations.

Which sets the design rule: deletion must be at least as reliable as insertion, and it must be driven by something that cannot forget. Not a webhook you might miss, not a queue that might drop a message — a source of truth you can diff.

Invalidating from a diff

For a codebase there is a perfect source of truth already: git knows exactly what changed between two states, including what was deleted and what was renamed.

So the index stores the commit it was built from, and updating is a diff against the current one:

// what changed since the indexed commit
const diff = await git.raw([
  "diff", "--name-status", "-M",     // -M detects renames
  indexedCommit, "HEAD",
]);

for (const line of diff.trim().split("\n")) {
  const [status, a, b] = line.split("\t");
  if (status === "D")            await dropChunksFor(a);
  else if (status.startsWith("R")) await renamePath(a, b);   // no re-embed needed
  else                            await reindexFile(b ?? a); // A or M
}
await setIndexedCommit(headSha);

Three properties make this hold up. It is complete — git cannot forget a file, so no deletion is missed. It is idempotent — re-running the same diff produces the same result, so a crashed job is retried rather than reconciled. And it is resumable — the stored commit is a checkpoint, so a process that dies halfway simply picks up from the last recorded position.

Rename detection is worth the flag. A moved file has identical content, so re-embedding it is pure waste; updating the path metadata is a cheap write.

Going finer than the file

File-level invalidation is a large improvement and still coarse. A thousand-line file with a one-line change re-embeds every chunk in it.

If chunks are already syntactic units — a function, a class, a method — you can hash each one and compare:

const incoming = new Map(
  chunksFrom(parse(source)).map((c) => [c.symbol, { ...c, hash: sha1(c.text) }])
);
const existing = await getChunkHashes(fileId);   // symbol -> hash

for (const [symbol, chunk] of incoming) {
  if (existing.get(symbol) !== chunk.hash) await upsertChunk(fileId, chunk);
}
for (const symbol of existing.keys()) {
  if (!incoming.has(symbol)) await dropChunk(fileId, symbol);   // removed
}

Now a commit touching one function re-embeds one chunk. In a repository where most commits are small, this is often an order of magnitude less embedding work than the file-level version, and the deletion path is exactly as reliable.

It also handles the case file hashing cannot: a function moved from one file to another is a delete plus an insert at file level, and at symbol level the hash matches, so it becomes a metadata update.

Changes that ripple past the thing that changed

The subtle failure. Some chunks are stale even though their own text did not change:

• A chunk enriched with context that came from elsewhere — the file header, the module docstring, the class it belongs to. Change the class name and every method chunk carrying it as context is stale.
• A summary or title generated for a document, when the document changed beneath it.
• Anything whose embedded text was assembled from more than one source.

Two honest options. Record the inputs each chunk was built from and invalidate on any of them changing — precise, and it means maintaining a small dependency graph. Or keep enrichment strictly local to the chunk so the problem cannot arise, accepting slightly less context per chunk.

For most systems the second is the better trade. A dependency graph you maintain by hand is a thing that will drift, and a drifted invalidation graph produces exactly the confident-and-wrong failure you were trying to eliminate.

Operating it

• Store the indexed commit in the same database as the chunks. If they can disagree, they will, and you will not know which is right.
• Make the update job idempotent and safe to run concurrently — take an advisory lock per repository so two runs cannot interleave into a half-updated state.
• Alert on index lag, meaning commits behind HEAD. It is the direct analogue of consumer lag, and it is the number that tells you the index is quietly going stale.
• Keep a full rebuild path and run it occasionally against a copy, comparing chunk counts and hashes with the incremental index. Incremental systems drift; scheduled verification is how you find out before a user does.
• Return the commit with every citation. Then an answer citing code that has since changed can be recognised as such rather than silently trusted.

The short version

A retrieval index over changing content is a synchronisation problem, and stale chunks fabricate with citations, which is worse than missing ones. Drive invalidation from something that cannot forget — for code, a git diff between the indexed commit and HEAD, with rename detection. Hash at the symbol level so a one-function change costs one embedding. Keep chunk enrichment local so nothing goes stale at a distance. Then alert on lag and reconcile against a full rebuild periodically.