skills.rest
logicalclocks
logicalclocksOFFICIAL
Explore all skills by this author →
Website𝕏@hopsworksGitHub

hops-monitoring

Monitor Hopsworks feature stores for drift and data quality issues.

15|32|Updated Nov 13, 2021
Search Tags:#drift detection#data validation#feature store#data monitoring#Hopsworks
One-click install
npx skills add https://github.com/logicalclocks/hopsworks-api --skill hops-monitoring
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hops-monitoring
Source: https://github.com/logicalclocks/hopsworks-api/tree/main/skills/hops/hops-monitoring
Command: npx skills add https://github.com/logicalclocks/hopsworks-api --skill hops-monitoring

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive solution for monitoring the Hopsworks feature store, enabling descriptive statistics, drift detection, data validation, alerts, and inference feature logging.

Core Features & Use Cases

  • Descriptive Statistics: Compute and track statistics on feature data.
  • Drift Detection: Monitor for data drift in feature groups and feature views.
  • Data Validation: Validate feature data using Great Expectations.
  • Alerts: Set up alerts for various events such as drift detection or validation failures.
  • Inference Feature Logging: Log inference features for post-hoc drift analysis.
  • Use Case: Use this Skill to ensure the integrity and quality of feature data in your Hopsworks project, helping to maintain data consistency and accuracy.

Quick Start

Use the hops-monitoring skill to set up monitoring for the 'amount' feature in the 'my_fg' feature group.

Frequently Asked Questions about hops-monitoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor data drift in a Hopsworks feature store?▼

You can monitor feature store drift by applying descriptive statistics and drift detection directly to feature groups and views. This requires a Hopsworks cluster with feature monitoring enabled and the Python SDK to track data quality issues.

Can I validate feature data using Great Expectations in Hopsworks?▼

Yes, feature data validation in Hopsworks integrates with Great Expectations to check data quality. This allows you to validate feature groups, set up alerts for validation failures, and ensure feature integrity for machine learning workflows.

What is needed to set up alerts for feature store data validation failures?▼

Alerts for validation failures require a Hopsworks cluster with feature monitoring enabled and the Python SDK. You configure alerts to trigger on drift detection or validation failures to maintain feature data integrity.

How do I log inference features for post-hoc drift analysis?▼

Logging inference features for post-hoc drift analysis involves recording inference data within your Hopsworks project. Using the Python SDK, you capture these features to analyze data drift after model predictions are made.

Do I need a Hopsworks cluster to track descriptive statistics on feature groups?▼

Yes, tracking descriptive statistics on feature groups requires a Hopsworks cluster with feature monitoring enabled. You use the Python SDK to compute and monitor these statistics within your feature engineering workflows.

When should I use data monitoring for machine learning feature pipelines?▼

Data monitoring should be used when you need to ensure the integrity and quality of feature data in machine learning pipelines. It helps maintain data consistency by detecting drift and validating features before model inference.

Related Skills

Explore More Data & Analytics Skills→
logicalclockslogicalclocks
15

hops-train

Train ML models with feature views and register them in the Hopsworks model registry.

Official
Intermediate
Sir-chawakornSir-chawakorn
9

monitor-ml-drift

Monitor production ML models for data drift and performance decay using statistical tests.

Community
Advanced
Sumeet138Sumeet138
1

data-quality-frameworks

Automate data quality validation across ETL/ELT pipelines with Great Expectations, dbt tests, and data contracts.

Community
Advanced
logicalclockslogicalclocks
15

hops-eda-checklist

Plan exploratory data analysis for Hopsworks feature-view training data.

Official
Intermediate