Claims Intake Knowledge Bases: 5 Freshness Rules for Versioned Retrieval

Short answer: treat every claim document as an immutable revision, and make retrieval select a valid revision before it selects a chunk. For an insurance marketplace building an internal knowledge-base bot, this keeps an adjuster from receiving a superseded coverage clause while still allowing an audit to reconstruct what the bot saw. The expensive part is rarely the embedding call. It is retention: storing every extracted chunk, vector, source pointer, and re-indexing attempt for documents that may be replaced minutes apart. I start by measuring that inventory, then decide which history must remain queryable and which history only needs an audit record. Where the bill and the risk actually come from A claims intake pipeline commonly keeps four copies of meaning: the original file, normalized text, chunk records, and index metadata. A fifth copy appears when a revised file is embedded before the old vectors are retired. At marketplace scale, that overlap dominates storage and reprocessing work more than the raw upload. The practical accounting unit is a revision, not a file. A 12-page medical invoice and its corrected 12-page invoice should have separate content hashes, extraction timestamps, policy identifiers, and effective intervals. Their chunks may have identical text, but they do not have identical legal context. Deduplicating by text alone can silently merge two valid revisions. I once assumed a single document_id plus an updated_at field would be enough. It wasn't. A late-arriving correction made an older chunk look current because ingestion time and policy effective time were different clocks. The fix was to store both clocks and require an explicit supersedes edge. Keep the raw artifact and a compact manifest for every revision, but keep vectors only for revisions that can be returned by the product's search policy. That is the first retention decision. When a revision is superseded, mark it non-current immediately, but do not delete it synchronously. Deletion can race with an in-flight answer, and the audit trail still needs the old hash. A tombstone plus a delayed garbage-collection job gives operators a deterministic boundary. How should insurance claims intake handle document versioning for retrieval? Use a two-stage query. Stage one resolves the admissible revision set; stage two ranks chunks inside that set. The filter should consider claim ID, document type, jurisdiction, policy effective date, and a revision status such as current or historical. Similarity is a ranking signal, not a time machine. Full stop. A useful record shape is deliberately boring: from dataclasses import dataclass from datetime import datetime @dataclass(frozen=True) class Revision: claim_id: str revision_id: str content_sha256: str effective_from: datetime effective_to: datetime | None supersedes: str | None status: str def admissible(revision: Revision, as_of: datetime) -> bool: return ( revision.status == "current" and revision.effective_from