Regex vs NLP for Legal PII Detection

Deciding whether to reach for a regular expression or a statistical NLP model to redact a given piece of legal PII comes down to whether the identifier has a fixed grammar or depends on context; getting that call wrong produces either brittle misses on structured IDs or blind spots on human names, and this page runs both engines on the same text to show exactly where each wins.

Prerequisites & version matrix #Permalink to this section

Regex handles what is grammatically fixed; NLP handles what is linguistically variable. The two are complementary, not competing, and the sections below quantify that on a shared fixture. For the broader decision framework see the parent guide on Detection Approach Trade-offs.

Step 1 — Deterministic detection with regex #Permalink to this section

Structured identifiers — Social Security numbers, Employer Identification Numbers, and docket/case numbers — follow rigid formats, so a compiled pattern catches them precisely and in sub-millisecond time.

import re

# Anchored patterns; \b boundaries avoid matching inside longer digit runs.
PATTERNS = {
    "US_SSN": re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
    "US_EIN": re.compile(r"\b\d{2}-\d{7}\b"),
    "CASE_NO": re.compile(r"\b\d{1,2}:\d{2}-[a-z]{2}-\d{3,6}\b", re.IGNORECASE),
}

def regex_spans(text: str):
    spans = []
    for label, pattern in PATTERNS.items():
        for m in pattern.finditer(text):
            spans.append((m.start(), m.end(), label))
    return spans

Step 2 — Contextual detection with spaCy NER #Permalink to this section

Person names, law-firm and corporate names, and the relationships that signal privilege have no fixed shape, so they require a model that reads surrounding language.

import spacy

# Load once at module scope; reuse across documents.
nlp = spacy.load("en_core_web_sm")

# Map spaCy's generic labels onto redaction entity types.
LABEL_MAP = {"PERSON": "PERSON", "ORG": "ORG", "GPE": "LOCATION"}

def ner_spans(text: str):
    doc = nlp(text)
    return [
        (ent.start_char, ent.end_char, LABEL_MAP[ent.label_])
        for ent in doc.ents
        if ent.label_ in LABEL_MAP
    ]

Step 3 — Compare on the same text #Permalink to this section

Running both over one passage exposes the division of labor: regex nails the identifiers spaCy ignores, and spaCy names the people regex cannot describe.

SAMPLE = (
    "Plaintiff Maria Alvarez (SSN 123-45-6789), an employee of "
    "Nordic Freight LLC (EIN 12-3456789), filed in case 1:24-cv-01822."
)

if __name__ == "__main__":
    print("regex:", regex_spans(SAMPLE))
    print("ner:  ", ner_spans(SAMPLE))
    # regex catches SSN 123-45-6789, EIN 12-3456789, case 1:24-cv-01822
    # ner catches Maria Alvarez (PERSON) and Nordic Freight LLC (ORG)

Compliance checkpoint #Permalink to this section

The controlling risk for redaction is under-redaction, not over-redaction. GDPR Article 17(1) grants data subjects the right to erasure, and a leaked identifier in a produced document is a failure of that obligation. Translate the right into an engineering control by setting a recall floor per entity type — for direct identifiers such as SSNs, that floor should be 1.0, achievable only because regex is exact on their format. Because NER recall on names is below 1.0, any document whose combined coverage dips under the floor must route to human review rather than auto-commit, so probabilistic gaps never become disclosed personal data.

Gotchas #Permalink to this section

• Regex misses formatting variants. 123 45 6789 or 123.45.6789 slips past a hyphen-only SSN pattern. Broaden the separator class ([-.\s]) and normalize whitespace before matching.
• NER misses rare surnames. Statistical models under-recognize uncommon or non-Western names and OCR-garbled tokens, dropping recall silently. Never rely on NER alone for a mandatory-erasure entity.
• Combine to raise recall. The two error sets barely overlap, so unioning regex and NER spans lifts total recall above either alone — the core reason production pipelines layer them.
• Overlap needs resolution. When both engines fire on crossing offsets, de-duplicate deterministically before masking, as shown in the parent guide, or the same text is redacted twice.

Verification & testing #Permalink to this section

This fixture asserts precision and recall per approach deterministically, without downloading a model, by scoring fixed prediction sets against a gold standard.

def score(pred: set, gold: set):
    tp = len(pred & gold)
    precision = tp / len(pred) if pred else 0.0
    recall = tp / len(gold) if gold else 0.0
    return round(precision, 2), round(recall, 2)

# Gold spans present in the sample passage.
GOLD = {(0, 0, "PERSON"), (0, 1, "ORG"),
        (0, 2, "US_SSN"), (0, 3, "US_EIN"), (0, 4, "CASE_NO")}

REGEX_PRED = {(0, 2, "US_SSN"), (0, 3, "US_EIN"), (0, 4, "CASE_NO")}
NER_PRED = {(0, 0, "PERSON"), (0, 1, "ORG")}

def test_regex_is_precise_but_partial():
    assert score(REGEX_PRED, GOLD) == (1.0, 0.6)

def test_ner_covers_the_remainder():
    assert score(NER_PRED, GOLD) == (1.0, 0.4)

def test_union_reaches_full_recall():
    assert score(REGEX_PRED | NER_PRED, GOLD) == (1.0, 1.0)

Running pytest confirms the headline claim: each engine alone caps out on partial recall, and only their union clears the erasure bar.

Related #Permalink to this section

This comparison sits under Detection Approach Trade-offs; pair it with the tooling-focused companion on Microsoft Presidio vs Custom spaCy Pipelines, then deepen each engine via Regex Rule Optimization for Legal Entities and spaCy NER for PII Detection. Part of the PII Detection & Automated Redaction Patterns field guide.