target-leakage-detection

Detect temporal, direct, and group data leakage in machine-learning feature sets.

14|2|Updated Feb 24, 2026
One-click install
npx skills add https://github.com/andikarachman/data-science-plugin --skill target-leakage-detection
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: target-leakage-detection
Source: https://github.com/andikarachman/data-science-plugin/tree/main/skills/target-leakage-detection
Command: npx skills add https://github.com/andikarachman/data-science-plugin --skill target-leakage-detection

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Detect data leakage that would inflate model performance during development but fail in production.

Core Features & Use Cases

  • Temporal leakage checks to ensure features are only available at prediction time.
  • Direct leakage detection to flag features that encode target information.
  • Statistical signals to surface suspicious feature behavior.
  • Group leakage checks to prevent train/test cross-contamination.
  • Remediation guidance to describe and fix leakage mechanisms.

Quick Start

Run a leakage audit on your dataset by applying the checks to identify features that leak target information before training.

Frequently Asked Questions about target-leakage-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is target leakage in machine learning and how do I detect it?

Target leakage happens when training features encode prediction-time information, inflating model performance. You detect it by validating temporal validity, checking target correlation, and ensuring train/test groups do not cross-contaminate.

How do I run a data leakage audit on my feature set before training?

Run a data leakage audit by applying temporal, direct, and group leakage checks to your dataset. This surfaces suspicious feature behavior and flags any variables that leak target information before model training begins.

How do I prevent temporal leakage in churn prediction models?

Prevent temporal leakage in churn prediction by enforcing temporal validity checks that ensure features are only available at prediction time, preventing future information from bleeding into historical training data.

What's the best way to fix target leakage in regulated analytics workflows?

Fix target leakage in regulated analytics by applying remediation guidance that describes leakage mechanisms, validating temporal and direct leakage, and holding out train/test groups to prevent cross-contamination.

Why does my model performance drop in production but not during training?

Model performance drops in production due to target leakage that inflates development metrics. Direct and temporal leakage checks identify features encoding target information, ensuring training data integrity matches real-world prediction conditions.

Can I use target leakage detection for pricing models with group-based data splits?

Yes, target leakage detection applies to pricing models by performing group leakage checks that prevent train/test cross-contamination, validating temporal feature availability, and flagging direct target correlation across grouped data splits.