Detection Approach Trade-offs

Choosing how to find personally identifiable information in legal documents is an engineering decision with compliance consequences: deterministic regex is exact on structured identifiers but blind to context, statistical named-entity recognition reads context but drifts, and neither alone clears a defensible recall bar. This guide, part of the PII Detection & Automated Redaction Patterns field guide, shows how to weigh, combine, and layer them.

Approach comparison matrix #Permalink to this section

No single detector satisfies both the precision needed to avoid over-redacting privileged text and the recall needed to avoid leaking identifiers. The matrix below summarizes where each approach earns its place, and why most production pipelines layer them rather than pick one.

Deterministic patterns should own anything with a fixed grammar, as covered in Regex Rule Optimization for Legal Entities. Free-text entities that depend on surrounding language belong to spaCy NER for PII Detection. The build-vs-buy axis is a distinct decision explored in the companion page on Microsoft Presidio vs Custom spaCy Pipelines.

Verified hybrid resolver #Permalink to this section

The value of layering is realized only if overlapping spans are reconciled deterministically. The resolver below merges regex anchors and NER spans, then de-duplicates overlaps with a stable priority order so identical inputs always yield identical redaction sets — a hard requirement for chain-of-custody reproducibility.

from dataclasses import dataclass
from typing import List, Literal

@dataclass(frozen=True)
class Span:
    start: int          # inclusive character offset
    end: int            # exclusive character offset (half-open interval)
    label: str          # e.g. "US_SSN", "PERSON"
    source: Literal["regex", "ner"]
    score: float        # 1.0 for deterministic regex anchors

def _overlaps(a: Span, b: Span) -> bool:
    # Half-open intervals share a character iff they cross.
    return a.start < b.end and b.start < a.end

def resolve(spans: List[Span]) -> List[Span]:
    # Deterministic ordering: earliest start, then widest span,
    # then regex before NER (structured anchors win ties), then score.
    priority = {"regex": 0, "ner": 1}
    ordered = sorted(
        spans,
        key=lambda s: (s.start, -(s.end - s.start), priority[s.source], -s.score),
    )
    kept: List[Span] = []
    for span in ordered:
        # Drop any candidate overlapping an already-accepted span.
        if any(_overlaps(span, k) for k in kept):
            continue
        kept.append(span)
    return kept

if __name__ == "__main__":
    # A regex SSN anchor and an NER span that partially overlap it.
    candidates = [
        Span(10, 21, "US_SSN", "regex", 1.0),
        Span(14, 30, "PERSON", "ner", 0.88),   # overlaps the SSN, dropped
        Span(48, 61, "PERSON", "ner", 0.91),   # disjoint, kept
    ]
    for s in resolve(candidates):
        print(f"{s.label:8} [{s.start}:{s.end}] via {s.source}")
    # US_SSN   [10:21] via regex
    # PERSON   [48:61] via ner

Compliance alignment #Permalink to this section

A redaction methodology is only defensible if it is documented and reproducible. Under FRCP Rule 34, producing parties must handle electronically stored information in a form that can be scrutinized; a hybrid detector whose span-resolution logic is version-controlled and deterministic lets you demonstrate exactly why each character was masked. The NIST Privacy Framework category PR.DS (Data Security) expects controls that protect data at rest and in transit throughout processing — recording which detector fired, its score, and the resolved outcome for every span satisfies that evidentiary expectation without ever persisting raw identifiers.

Performance and scale #Permalink to this section

The two approaches sit on opposite ends of the cost curve. A compiled regex evaluates a candidate in the sub-millisecond range, so a bank of a few dozen patterns adds negligible latency to ingestion. Statistical NER costs roughly milliseconds per token on CPU; a 3,000-token filing can take 200–400 ms on en_core_web_sm and several times that on a transformer pipeline. The practical pattern is to run regex first as a cheap, high-precision anchor pass, then reserve NER for the residual free text, keeping combined throughput near 6,000–9,000 pages/hour on 4 vCPUs while confidence routing, described in Confidence Threshold Configuration, gates the borderline cases.

Edge cases & failure handling #Permalink to this section

• Overlapping spans double-redact. When regex and NER both fire on adjacent-but-crossing offsets, naive concatenation masks the same text twice and corrupts downstream offset math. Remediation: run every candidate through the deterministic resolver above so a stable priority order picks one span per region.
• NER drift. Model accuracy degrades as document corpora evolve (new firm styles, redacted training leakage), silently lowering recall. Remediation: track precision/recall against a quarterly gold set and alert when either crosses a floor, then retrain or recalibrate.
• Regex catastrophic backtracking. A poorly bounded pattern with nested quantifiers can pin a CPU for seconds on an adversarial input, stalling the queue. Remediation: prefer possessive/atomic constructs and anchored character classes, cap input length per match, and unit-test patterns against long non-matching strings.
• Silent under-redaction. Both engines missing a rare identifier produces a false negative with no error. Remediation: enforce a combined recall floor and route low-coverage documents to human review rather than auto-committing.

Related #Permalink to this section

Continue with the companion pages on Regex vs NLP for Legal PII Detection and Microsoft Presidio vs Custom spaCy Pipelines, then tune the individual engines through Regex Rule Optimization for Legal Entities, spaCy NER for PII Detection, and Confidence Threshold Configuration. Part of the PII Detection & Automated Redaction Patterns field guide.