In the first article of the "modular knowledge" series, we put knowledge on a shelf: small, labeled boxes an agent can pick from, instead of one pile it has to read through. That answers where the right piece is.

It does not answer whether what sits in the box is still true.

The paperwork that became a burden

A few days ago, I was sorting through old paperwork when I found documents I had not looked at in years.

Some had once been indispensable. They had helped me decide, solve problems, find my way when the right information mattered. Now they were gathering dust.

Their content had not changed. Their relevance had.

Some were simply obsolete. Others still held history. A few were worse: outdated, yet easy to mistake for something current. They were no longer just useless. They had become a burden, because they are misleading, making it harder to tell what still matter.

Data does not need to disappear to stop creating value. Sometimes its continued existence is the problem.

Three documents, one question

Now move that pile of paperwork into a company.

A global policy lead asks the company's AI assistant a simple question:

what is our requirement on a given regulatory topic?

The search returns three documents.

1. A global policy, approved two years ago.
2. An updated guidance, published six months after it.
3. A regional interpretation, written for one regulatory context.

All three contain relevant information. All three are correct in their own context. But only one is the reference for this question.

And the right one depends on the question. Ask what the requirement is today, and the updated guidance wins. Ask what the requirement was when a decision was made two years ago, and the original policy is the right source. Ask what the requirement is for this region, and the third one wins.

Same three documents, three different answers.

If the system cannot tell which is current, the retrieval (the step where the system looks up the pieces it will use) stops being an advantage. It becomes a liability.

The answer that sounded right

The assistant answers from the global policy approved two years ago.

Why? Because it is the closest match to the question. It uses the same words, it is complete, it is well written. The updated guidance is shorter and more technical, so it ranks lower. The answer comes back fluent, structured and confident.

And wrong.

Nothing crashed. No alert fired. A document the organization had moved past walked straight into an answer, because nothing told the system it was no longer alive. Call it a zombie: a document retired in the real world that keeps walking into answers in the digital one.

This is not a bug in a particular tool. A vector search (finding passages whose meaning resembles the question) can retrieve an outdated policy precisely because it is so similar to the question. A language model can summarize a superseded document convincingly. Neither knows the document has lost its authority.

Similar is not current.

And the better your similarity search, the more convincingly it brings back the dead.

Who used to catch this

Until now, a person did.

The colleague who had been around for years knew which of the three documents counted, and quietly ignored the others. That knowledge lived in their head, not in the files.

Ask why the guidance outranks the policy, and the answer is rarely written down: "I was in the room when it changed." Even a stranger opening an old PDF sees a date in the footer, hesitates, and asks someone down the hall.

An agent has no footer reflex and no one down the hall. It has the words of your question and the words of your documents.

This is a quiet form of knowledge loss. When experienced people move on or teams are reorganized, the organization keeps its documents and loses the context needed to use them.

Labels (metadata: structured facts about a document, such as its status) move that context out of heads and into the files, where an agent can read it.

"Just delete the old ones"

The obvious fix comes quickly: delete them.

It is also the wrong one, or at least not the whole answer. Deleting the zombie also deletes the evidence. Months later, an auditor, a regulator or a colleague asks what the requirement was when a past decision was made. The old version is the only proof.

The same policy that is wrong for today's question is indispensable for last year's. You cannot simply kill what you may need to prove.

Three decisions hiding in one

The mistake is to treat "is this old document still good?" as a single question. It is three.

1. Validity: is the information still true for a given use?
2. Retention: must we keep it, and for how long?
3. Disposition: when we no longer have to, what happens to it (archive, review, securely delete)?

A single status: obsolete cannot hold all three. A superseded policy can be invalid for today's decisions, retained for audit, and protected from deletion, all at once.

Retention rules vary by country, sector and type of record, so the point is not to prescribe periods. It is to keep the three decisions apart.

Why keep a retired document at all? For four reasons that can coexist. It may still support a current process. It may prove what was required at a given moment. It may explain how a position evolved. Or it may simply give context, even though it is no longer the authority.

Authority is also contextual. A regional interpretation is authoritative in its region and inappropriate as the source for a global decision. Its validity is not in question. Its scope is.

And age does not settle anything. An old document can be essential evidence. A recent one can already be obsolete if it was written on top of superseded guidance.

The age of a document does not determine its value. Its relevance to a question does.

Label it, fence it

So we do not bury the zombie. We fence it, and we put a headstone on it: the facts about what it was, until when it was valid and what replaced it.

On the shelf from the first article, this is the back row. Boxes no longer in daily use, still standing, each with a tag.

Documents have a lifecycle: alive (approved), retired (superseded), archived

The status field says which.

Here is what a headstone 🪦 can look like, in a YAML header (a few lines of plain text at the top of a file, as in the first article; illustrative, adapt it to your own governance):

---
title: Global policy on product quality
status: superseded          # approved | superseded | archived
valid_from: 2024-11-01
valid_until: 2025-05-14
superseded_by: guidance-product-quality-2025.md
scope: global
retention: keep for audit   # set by your records rules, not by this file
---

Three things follow from it. The document stays retrievable for history questions. It is no longer eligible as the current reference. And it points to its replacement, so anyone (or any agent) that lands on it knows where to go.

Retrieval then changes order. It filters on status and scope first, and only then ranks by similarity: current, global, approved; then the closest match. A history question flips the filter: what applied in March last year?

These are different tools, and it helps to keep them apart.

• Full-text search finds documents by their words.
• Metadata filtering narrows results by structured properties such as status or scope.
• Semantic search finds meaning even when the wording differs.
• RAG (retrieval-augmented generation) hands the retrieved passages to a language model that writes the answer.

They complement each other, but none of them knows about authority unless something tells it: labels, or the version control of a document system.

Add semantic search or RAG to an unlabeled collection and you have not solved validity; you have automated its failure, faster.

A policy typically moves through drafting, review, approval, active use, supersession and archive. Each step has consequences. Approval makes a document authoritative. A new version supersedes the previous one. A change in regulation can trigger a review. The end of a retention period starts an assessment of what to do next. Some of these steps can be automated; others need an accountable person.

Lifecycle changes are triggered by events, but an event is not a verdict. A review date triggers a review; it does not make a document false. Publishing a replacement can automatically mark the old one as superseded. Retiring, archiving or deleting needs a decision by someone accountable, with a record of why.

One honest note before reaching for new technology. A plain document management system (software that stores documents with versions, metadata and full-text search) already solves a large part of this.

If the question is "which approved policies does this function own?", a structured query is more reliable, and easier to audit, than a semantic search.

AI does not remove the need for disciplined information management. It raises the stakes.

Back to the submission

Take the regulatory submission from the first article. A regulatory submission is the file a company sends to a health authority to obtain or keep approval for a medicine. It is assembled from many documents written by different teams.

Consider the agent assembling a section on a manufacturing requirement. The newest guidance is the reference. But the older document is longer, more detailed and closer to the wording of the request, so the agent ranks it first. The section reads perfectly. The superseded source is even cited correctly, with a real reference, in the right place.

Nobody notices until a reviewer, late in the process, spots a requirement that no longer applies. Or nobody does.

Catching it late means reopening the section, re-checking everything built on it, and re-running the review. Not catching it means a file that is internally consistent and externally wrong.

With the headstone and the filter, the older document would not have been eligible as current. The agent would have flagged a superseded source and pointed to its replacement. A deterministic check, such as "no superseded source in a current submission", is a cheap safeguard. People still review. But they review a file that starts from the right source.

A person reviewing three documents may sense that something is off. An agent drawing on a very large collection will not, and it will do so at machine speed, section after section.

Left alone, the problem grows. Every version that is not marked, every replaced guidance still looking current, adds another zombie. As the knowledge base grows, the share of it that is still alive can shrink without anyone noticing, and the agent has no way to know.

Proportion, not perfection

Not every document deserves a headstone. A meeting note from last spring and a policy that drives a regulated filing do not need the same care.

Start where a wrong answer costs the most and where the question gets asked the most, and let the rigor follow the risk.

Over-labeling everything is its own failure: a lifecycle nobody maintains is just another stale shelf.

What this means for you

If you keep three things from this article, keep these:

• Retrieval finds what is similar, not what is still true.
• Do not delete, do not ignore: fence. Validity, retention and disposition are three decisions, not one.
• Filter on status and scope before ranking by similarity.

And here is your next Monday step, small enough to finish in an afternoon: take the ten documents your team queries most. Give each one a name for who looks after it (the next article is about that; for now, a name is enough), a status and a valid_until (or a review date if it is still current).

Then ask your agent ten questions those documents answer, and count how many answers cite a superseded source. That count is your baseline.

One question remains, and we skipped it on purpose. Someone has to decide that a document is dead. Who?

For now, one question to sit with: if your agent quoted a superseded document tomorrow, who would notice, and how long would it take?

Find more on the "Modular Knowledge" series

• Part 1/4: The differentiator is the knowledge, not the LLM
• Part 2/4 (this article): How to handle outdate content
• Part 3/4 (coming soon): Knowledge ownership & lifecycle
• Part 4/4 (coming soon): Your knowledge base is a data product

Follow me on LinkedIn