skills.rest
robotijn
robotijnCOMMUNITY
Explore all skills by this author →
GitHub

feature-store-validator

Audit feature store configurations for training-serving skew and data leakage.

3|2|Updated Jan 23, 2026
Search Tags:#mlops#model validation#feature store#point-in-time join#training-serving skew#feature drift#feature lineage
One-click install
npx skills add https://github.com/robotijn/ctoc --skill feature-store-validator
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: feature-store-validator
Source: https://github.com/robotijn/ctoc/tree/main/skills/data-ml/feature-store-validator
Command: npx skills add https://github.com/robotijn/ctoc --skill feature-store-validator

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates the risk of silent feature store failures that cause production machine learning models to underperform or fail without warning in offline validation metrics. Uncaught issues like training-serving skew, point-in-time data leakage, and unmonitored feature drift can lead to incorrect predictions, financial loss, and damaged user trust before teams detect the problem.

Core Features & Use Cases

  • Critical Failure Mode Auditing: Scans feature stores for 11 high-impact failure modes including training-serving skew, missing point-in-time joins for training data, online/offline value inconsistency, unmonitored feature drift, missing feature ownership, and unpinned feature versions at inference.
  • Cross-Platform Validation: Supports auditing for Feast, Tecton, Hopsworks, Vertex AI Feature Store, Databricks Feature Store, and AWS SageMaker Feature Store, with platform-specific best practice checks for each.
  • Use Case: For a team deploying a fraud detection model, this Skill would catch if the training pipeline uses a naive join that leaks future transaction data into features, or if the online feature store values don't match offline training values, preventing silent model degradation in production.

Quick Start

Invoke the feature-store-validator skill to audit your feature store for critical flaws that could cause production ML model failures.

Frequently Asked Questions about feature-store-validator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect point-in-time data leakage in my feature store training pipelines?▼

To detect point-in-time data leakage in feature store training pipelines, audit feature definitions for mandatory point-in-time joins to prevent future data from leaking into features and causing silent production ML failures.

Why does my ML model degrade in production when offline validation metrics look fine?▼

Your ML model degrades in production due to training-serving skew, online/offline value inconsistency, or unmonitored feature drift. These silent feature store failures bypass offline metrics and require dedicated configuration auditing to detect.

Does this feature store validation support Feast, Tecton, and Hopsworks?▼

Yes, feature store validation supports cross-platform auditing for Feast, Tecton, Hopsworks, Vertex AI, Databricks, and AWS SageMaker Feature Store, applying platform-specific best practice checks to catch critical failure modes.

What's the best way to enforce single transform paths for online and offline feature computation?▼

The best way to enforce single transform paths for online and offline feature computation is auditing feature definitions against 2026 MLOps best practices to prevent training-serving skew by ensuring identical transformation logic across both environments.

How do I audit feature definitions for unpinned feature versions at inference?▼

To audit feature definitions for unpinned feature versions at inference, scan feature store configurations for explicit versioning to prevent silent model failures by ensuring inference always uses the correct, pinned feature version.

When should I run scheduled consistency checks for online and offline feature stores?▼

You should run scheduled consistency checks for online and offline feature stores as part of your regular MLOps workflow to enforce per-feature freshness SLAs and detect value inconsistency before production ML models degrade.

Related Skills

Explore More Data & Analytics Skills→
ivanshamaevivanshamaev
14

feature-store

Build a Feast feature repository with Entities, FeatureViews, and FeatureServices.

Community
Advanced
olehsvyrydovolehsvyrydov
13

mlops-engineer

Automate design, deployment, and operation of production ML systems.

Community
Advanced
aiunlocked1412aiunlocked1412
14

ml-engineer

Design and deploy production ML systems with MLOps pipelines for serving and monitoring.

Community
Advanced
Tyler-R-KendrickTyler-R-Kendrick
1

hf-cloud-python-env-setup

Create isolated Python environments for SageMaker and AWS automation tasks.

Community
Intermediate