An Open-Source PHI Scrubber That Keeps Patient Data Local by Stacking an LLM With Old-School Pattern Matching

DeidSharp layers a local 26B LLM with regex and clinical NER to catch protected health info without sending text off-site. Beats or matches Stanford/Penn on two PhysioNet corpora.

Imagine you're proofreading a letter for a friend who's paranoid about leaving their address anywhere. You read through once with a highlighter, catching obvious stuff — phone numbers, dates, ZIP codes. Then you hand it to a second reader who knows medical jargon and catches the doctor names you missed. Then a third reader — slower, smarter, more contextual — goes through and finds the one reference to "the clinic on Elm Street" that the first two missed. Now stack all three sets of highlights. Anything ANY reader flagged gets redacted. That's DeidSharp: three detectors (regex patterns, a clinical NER model, a local 26B mixture-of-experts LLM), unioned together, fail-closed on any detector error. The committed claim: a locally hosted LLM stacked with deterministic and neural detectors finds more protected health information than existing open-source de-identification tools, at controllable precision, without sending patient text to a cloud API. This is not a new architecture or a new model — it's a new stacking strategy applied to an operational problem that matters enormously in healthcare IT. The ladder here is honest and well-constructed. DeidSharp was compared against five named baselines — deid, Philter, Stanford/Penn, OpenAI Privacy Filter, and GLiNER — on reporting halves of two PhysioNet corpora (nursing notes with 889 spans, MIMIC-IV-Note discharge summaries with 29,023 spans) plus synthetic ASQ-PHI queries. The recall-first stack hit 98.9% on nursing notes (+2.8 over Stanford/Penn) at 9.3 points lower precision. The precision-first stack matched Stanford/Penn recall (96.5%) at 19.6 points higher precision (83.2%). On discharge summaries, both stacks led Stanford/Penn by 20+ recall points. But — and this is the kind of honesty you want — no configuration beat deid significantly on deid's home corpus, and Philter found more discharge-summary spans than the recall-first stack, albeit at 23 points lower precision. The paper doesn't hide its losses. Architecturally, this is an ensemble of three detector families: deterministic pattern matchers (regex, dictionaries), a clinical named-entity recognition model (neural, supervised), and a 26B mixture-of-experts LLM running locally. The key design choice is the fail-closed union: if any detector flags a span, it's flagged; if any detector errors out, the pipeline halts rather than silently missing PHI. The LLM does the heavy lifting. The NER mostly adds place names. An interesting finding: when asked to verify its own findings, the LLM discarded a quarter of true PHI — self-verification hurts recall, which is exactly the kind of failure mode you want documented. Integrity is above average for this kind of systems paper. The protocol was prespecified. Configurations were fixed on tuning halves and evaluated on reporting halves. Comparisons used paired patient-level bootstrap confidence intervals. All four prespecified comparisons held. The corpora are public (PhysioNet). The code is on GitHub. The main weakness: this is still one-team evaluation on public benchmarks, not independent replication on private clinical data from a different institution. The milestone that matters is deployment on real EHR text at hospital scale. DeidSharp demonstrates viability on PhysioNet data; the gap is institutional validation on private clinical notes where PHI distributions differ, annotation standards vary, and the consequences of a miss are real. The paper's 98.9% recall on nursing notes means roughly 1 in 100 PHI spans still leaks — for a 500-bed hospital generating thousands of notes daily, that's a non-trivial residual risk that needs quantification in situ. The obvious next experiment not run: testing on private, multi-institution clinical text with genuinely unseen PHI distributions. The honest read is (a) — access to such data requires IRB approvals and institutional partnerships that a single-author open-source project doesn't easily command. This is a resource constraint, not a hidden failure.