skills.rest
maziyarpanahi
maziyarpanahiCOMMUNITY
Explore all skills by this author →
Website𝕏@maziyarpanahiGitHub

gating-deid-leakage

Fail CI when PHI recall drops below approved thresholds or critical identifiers leak.

5.0k|615|Updated Oct 4, 2025
Search Tags:#pytest#github actions#regression testing#de-identification#release gating#ci gate#phi leakage
One-click install
npx skills add https://github.com/maziyarpanahi/openmed --skill gating-deid-leakage
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gating-deid-leakage
Source: https://github.com/maziyarpanahi/openmed/tree/main/skills/gating-deid-leakage
Command: npx skills add https://github.com/maziyarpanahi/openmed --skill gating-deid-leakage

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill prevents weakened clinical de-identification from reaching production by failing CI when PHI recall drops below policy thresholds or critical identifiers leak.

Core Features & Use Cases

  • Recall Regression Gates: Enforce per-label recall floors against a synthetic held-out PHI evaluation set.
  • Zero-Leakage Assertions: Fail closed when any critical identifier, such as an SSN or credit-card number, is detected.
  • CI Integration: Wire pytest or the OpenMed release-gate CLI into GitHub Actions and block merges when evaluation errors, quarantined reports, or failed gates occur.
  • Use Case: Add a required pull-request check that evaluates a candidate de-identification model, compares it with a pinned baseline, and prevents release when leakage or recall regressions are found.

Quick Start

Use the gating-deid-leakage skill to add a required GitHub Actions check that runs the OpenMed de-identification suite and fails on critical leakage or recall below the approved floor.

Frequently Asked Questions about gating-deid-leakage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I block de-identification regressions in GitHub Actions before merging?▼

To block de-identification regressions in GitHub Actions, you wire a pytest gate or release-gate CLI into a required pull-request check that fails CI when critical identifiers leak or PHI recall drops below an approved threshold.

What is a recall floor validation for clinical de-identification?▼

Recall floor validation for clinical de-identification is a release gating mechanism that enforces per-label PHI recall minimums against a held-out synthetic evaluation set, failing closed if the model drops below approved policy thresholds.

How do I set up zero-leakage assertions for PHI in pytest?▼

Setting up zero-leakage assertions for PHI in pytest involves configuring fail-closed evaluations that immediately fail CI when any critical identifier, such as an SSN or credit-card number, is detected in the de-identified output.

Can I use a pinned baseline model to prevent PHI leakage in CI?▼

Yes, you can use a pinned baseline model to prevent PHI leakage in CI by comparing candidate de-identification models against the pinned milestone, blocking merges when evaluation errors or recall regressions occur.

Does the OpenMed ReleaseGate integrate with pytest for release gating?▼

Yes, the OpenMed ReleaseGate integrates with pytest by applying release gating checks that evaluate held-out synthetic PHI, ensuring fail-closed validation and merge-blocking when quarantined reports or failed gates occur.

Related Skills

Explore More Software Engineering Skills→
maziyarpanahimaziyarpanahi
5.0k

evaluating-with-leakage-gates

Evaluate de-identification and clinical NER models for residual PHI leakage before release.

Community
Advanced
tiancaiamaotiancaiamao
5

land

Merge approved GitHub pull requests after CI checks pass and update task state.

Community
Advanced
brazil-benchbrazil-bench
11

re-evaluate

Detect new commits and closed issues, then re-run evaluations and update reports.

Official
Advanced
maziyarpanahimaziyarpanahi
5.0k

auditing-deid-leakage

Detect residual identifiers in de-identified clinical text and block release.

Community
Advanced