databricks-unity-catalog

Query Unity Catalog system tables and manage volumes in Databricks.

Updated Mar 23, 2024

Search Tags:#billing #lineage #compute #databricks #unity catalog #system tables

Ready to run

npx skills add https://github.com/m19c/dotfiles --skill databricks-unity-catalog-m19c

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-unity-catalog
Source: https://github.com/m19c/dotfiles/tree/main/claude/.claude/skills/databricks-unity-catalog
Command: npx skills add https://github.com/m19c/dotfiles --skill databricks-unity-catalog-m19c

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies access and querying of Unity Catalog system tables and volumes, making it easier to audit, analyze lineage, monitor billing, track compute resources, review job execution, and profile data quality.

Core Features & Use Cases

System Tables: Query and manage Unity Catalog system tables (audit, lineage, billing, compute, jobs, query history).
Volumes: Manage file operations and permissions on Unity Catalog volumes.
Data Profiling: Create quality monitors on Unity Catalog tables to track data profiles and detect drift.
Use Cases: For instance, analyze DBU consumption by workspace for cost optimization or set up a monitor to track lineage changes in your datasets.

Quick Start

Run the mcp__databricks__execute_sql tool with a query against the system tables.

Frequently Asked Questions about databricks-unity-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I query Databricks Unity Catalog system tables for audit logs?▼

To query Databricks Unity Catalog system tables for audit logs, you can run SQL queries against the system tables using the execute_sql tool. This simplifies access to audit data, lineage tracking, and system-level monitoring within your Databricks environment.

Can I track Databricks billing and compute resource usage by workspace?▼

Yes, you can track Databricks billing and compute resource usage by querying the Unity Catalog billing and compute system tables. This allows you to analyze DBU consumption and optimize costs across your workspaces.

How do I monitor data quality and detect drift on Unity Catalog tables?▼

You can monitor data quality and detect drift on Unity Catalog tables by creating quality monitors. This feature tracks data profiles and identifies changes in your datasets over time.

What is the best way to analyze data lineage in a Databricks environment?▼

The best way to analyze data lineage in a Databricks environment is by querying the Unity Catalog lineage system tables. This simplifies tracking how data flows through your pipelines and reviewing lineage changes.

Does this approach support managing file permissions on Unity Catalog volumes?▼

Yes, this approach supports managing file permissions on Unity Catalog volumes. You can perform file operations and manage access controls directly through the Unity Catalog interface.