anomaly-detection

Detect schema-level anomalies for table freshness and completeness in Databricks Unity Catalog.

5|6|Updated Jan 26, 2026
One-click install
npx skills add https://github.com/databricks-solutions/vibe-coding-workshop-template --skill anomaly-detection-databricks-solutions
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: anomaly-detection
Source: https://github.com/databricks-solutions/vibe-coding-workshop-template/tree/main/data_product_accelerator/skills/monitoring/04-anomaly-detection
Command: npx skills add https://github.com/databricks-solutions/vibe-coding-workshop-template --skill anomaly-detection-databricks-solutions

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automatically monitors your Databricks tables for unexpected changes in freshness and completeness, alerting you to potential data pipeline issues before they impact downstream consumers.

Core Features & Use Cases

  • Schema-level Monitoring: Applies freshness and completeness checks across all tables within a Databricks schema.
  • Automated Anomaly Detection: Uses ML models to detect stale or incomplete tables without manual metric definition.
  • Use Case: Ensure your critical Gold layer tables are always up-to-date and contain the expected volume of data, preventing reporting errors and business disruptions.

Quick Start

Enable anomaly detection on the 'sales' schema in the 'prod' catalog.

Frequently Asked Questions about anomaly-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor data freshness and completeness in Databricks Unity Catalog?

You can monitor Databricks Unity Catalog data freshness and completeness by applying schema-level anomaly detection to automatically track table updates and data volume using ML models.

How do I set up automated alerts for stale or incomplete Databricks tables?

To configure alerts for stale or incomplete Databricks tables, enable automated anomaly detection on your schema to identify data pipeline issues and alert downstream consumers before disruptions occur.

Does Databricks Data Quality API support automated anomaly detection without manual metrics?

Yes, the Databricks Data Quality API supports automated anomaly detection by using built-in ML models to evaluate table freshness and completeness without requiring manual metric definition.

What is the best way to ensure Gold layer tables are up-to-date and contain expected data?

The best way to ensure Gold layer tables remain up-to-date and contain expected data volumes is applying schema-level anomaly detection to monitor baseline data reliability and prevent reporting errors.

Can I apply data quality monitoring across all tables within a specific Databricks schema?

Yes, you can apply data quality monitoring across all tables within a specific Databricks schema by enabling schema-level anomaly detection checks for continuous freshness and completeness tracking.