Redaction Version Tracking

A redacted document is not a single artifact but a sequence of states, and defensibility depends on being able to name and prove every one. Redaction version tracking records each state a file passes through — original, detected, redacted, reviewed — as an immutable, content-addressed snapshot, giving the Immutable Audit Trails & Chain of Custody layer the ability to replay or roll back any decision on demand.

Why Versions Must Be Content-Addressed #Permalink to this section

Sequential integer version numbers carry no proof. Anyone with write access can renumber, reinsert, or quietly replace v3, and nothing in the record objects. Content addressing removes that trust assumption: a version’s identity is the SHA-256 digest of the fields that define it, so its name cannot be forged without changing the thing it names. Formally, a version is the tuple (parent version id, content hash, redaction decision set, actor). The parent id threads each state to the one before it, forming a lineage that mirrors the hash-linked record built in Cryptographic Audit Manifests; the content hash pins the exact bytes at that state; the decision set records which spans were removed and why; the actor names the operator or service that produced it.

Because every field is captured, snapshots are deterministic and replayable. Given the original bytes and a decision set, re-running the pipeline must yield the identical redacted content hash. That property is what lets you roll a matter back to its reviewed state after a faulty rule deploy, or replay the exact sequence an opposing party disputes, without keeping a mutable “current” pointer that could drift.

Version State & Configuration #Permalink to this section

Each state in the lifecycle is a distinct version with fixed semantics. The table below defines the tracked fields and their production defaults.

Content-Addressed Version Store #Permalink to this section

The store keys versions by their derived content_id and blobs by content_hash, so re-storing an identical state is idempotent and shared bytes are physically deduplicated.

import hashlib
import json
from dataclasses import dataclass
from typing import Optional

@dataclass(frozen=True)
class Version:
    """One immutable state of a document in the redaction lifecycle."""
    parent_id: Optional[str]   # content id of the prior version; None for the root
    stage: str                 # original | detected | redacted | reviewed
    content_hash: str          # SHA-256 of the document bytes at this state
    decisions_hash: str        # SHA-256 of the canonical redaction decision set
    actor: str                 # operator ID or service account (never raw PII)

    def content_id(self) -> str:
        """Deterministic identity: SHA-256 over the fields that define the version."""
        material = json.dumps({
            "parent_id": self.parent_id,
            "stage": self.stage,
            "content_hash": self.content_hash,
            "decisions_hash": self.decisions_hash,
            "actor": self.actor,
        }, sort_keys=True, separators=(",", ":")).encode("utf-8")
        return hashlib.sha256(material).hexdigest()


class VersionStore:
    """Content-addressed store where identical states collapse to one entry."""

    def __init__(self) -> None:
        self._versions: dict[str, Version] = {}   # content_id -> Version
        self._blobs: dict[str, bytes] = {}        # content_hash -> bytes (dedup)

    def put(self, version: Version, blob: bytes) -> str:
        # Guard: the declared content hash must match the bytes we were handed.
        if hashlib.sha256(blob).hexdigest() != version.content_hash:
            raise ValueError("content_hash does not match blob bytes")
        # Guard: a non-root version must point at a parent we already hold.
        if version.parent_id is not None and version.parent_id not in self._versions:
            raise KeyError(f"orphaned parent pointer: {version.parent_id}")
        cid = version.content_id()
        # Idempotent write: re-storing the same state is a no-op, not a duplicate.
        self._versions.setdefault(cid, version)
        # Blob dedup: N versions sharing bytes keep a single physical copy.
        self._blobs.setdefault(version.content_hash, blob)
        return cid

    def lineage(self, content_id: str) -> list[Version]:
        """Walk parent pointers back to the root, for replay or rollback."""
        chain: list[Version] = []
        cursor: Optional[str] = content_id
        while cursor is not None:
            node = self._versions[cursor]
            chain.append(node)
            cursor = node.parent_id
        return list(reversed(chain))

Compliance Alignment #Permalink to this section

Version tracking discharges two obligations at once. Under FRCP Rule 34, a producing party must be able to render responsive electronically stored information in a reasonably usable form and account for how it was processed; a content-addressed lineage lets counsel demonstrate the exact original, the redactions applied, and the reviewed output as separate, provable states rather than a single opaque file. Under GDPR Article 30, controllers must maintain records of processing activities — the actor, the categories of data acted on, and the operation performed. Because every version binds an actor and a decisions_hash to an immutable id, the store is that record of processing, queryable per document without retaining the personal data itself.

Performance & Scale #Permalink to this section

Deduplication by content_hash is what keeps immutability affordable. A discovery production commonly moves a 40 MB PDF through four states, but the detected state changes only annotation metadata, not the underlying page bytes. Naively persisting every state stores 160 MB per document; hashing the blobs first means the unchanged 40 MB body is written once and the three later states reference it, cutting stored volume to roughly 40 MB plus small decision sets. Across a 10,000-document matter that is the difference between 1.6 TB and about 0.4 TB, before any object-store compression. The digest computation costs one SHA-256 pass per state — a few hundred milliseconds for a 40 MB file — which is negligible against parsing and rendering.

Edge Cases & Failure Handling #Permalink to this section

• Non-deterministic redaction breaks dedup. If the redaction renderer embeds a timestamp, random object id, or unsorted annotation order, two runs of the same decision set produce different bytes and therefore different content_hash values, defeating deduplication and, worse, making replay unverifiable. Remediation: strip volatile metadata and canonicalize output ordering before hashing, so identical inputs yield identical bytes.
• Orphaned parent pointer. A version whose parent_id references a record that was never committed — or was pruned by an overzealous retention job — leaves lineage unwalkable. The store rejects such writes with KeyError at put time; operationally, never delete a version that any child still references, and route orphan-detection failures to quarantine rather than silently dropping them.
• Concurrent version writes. Two workers reviewing the same document may both attempt to append a child of the same parent, racing to create divergent branches. Because identity is content-derived, identical states collapse harmlessly, but genuinely different decision sets create a fork. Serialize writes per document id with an advisory lock, or reconcile forks through the Human-in-the-Loop Override Sync path so a reviewer chooses the authoritative branch.

Related #Permalink to this section

Go deeper with Deterministic Diffing of Redaction Versions to compute exactly which spans changed between two states, and Reconstructing Audit Timelines from Event Logs to render a document’s full history for review. Version identities anchor the hash-linked records in Cryptographic Audit Manifests, and forks are resolved through Human-in-the-Loop Override Sync. Part of the Immutable Audit Trails & Chain of Custody field guide.