skills.rest
Blackkadder
BlackkadderCOMMUNITY
Explore all skills by this author →
GitHub

querying-mlflow-metrics

Fetch aggregated trace metrics from MLflow tracking servers.

Updated Mar 19, 2026
Search Tags:#monitoring#llm#metrics#aggregation#latency#mlflow#token-usage
One-click install
npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill querying-mlflow-metrics-blackkadder
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: querying-mlflow-metrics
Source: https://github.com/Blackkadder/databricks-apps-and-agents-workshop/tree/main/.claude/skills/querying-mlflow-metrics
Command: npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill querying-mlflow-metrics-blackkadder

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Provides a straightforward way to collect and aggregate trace-level metrics from MLflow tracking servers so teams can monitor token usage, latency, trace counts, and evaluator quality scores across experiments without building custom queries.

Core Features & Use Cases

  • Aggregated Metrics: Compute COUNT, SUM, AVG, MIN, MAX and percentiles (P50, P95, P99) for trace and span metrics across specified experiments.
  • Flexible Grouping and Time Buckets: Group results by dimensions such as trace name or status and bucket by time interval for hourly or daily trend analysis.
  • LLM Cost & Quality Analysis: Summarize token usage and latency for cost estimation, and compute assessment statistics to monitor quality evaluations.
  • Use Case: Monitor hourly token consumption across LLM traces to identify cost spikes and slow endpoints over the last 24 hours.

Quick Start

Run the fetch_metrics.py script with your MLflow server URL, experiment IDs, metric, aggregations, and optional time window to retrieve aggregated metrics and trends.

Frequently Asked Questions about querying-mlflow-metrics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I aggregate MLflow trace metrics for LLM token usage and latency analysis?▼

To aggregate MLflow trace metrics for token usage and latency, run the fetch_metrics.py script with your MLflow server URL, experiment IDs, metric, and aggregation parameters to output table or JSON summaries.

What aggregation functions can I compute for trace metrics across MLflow experiments?▼

You can compute COUNT, SUM, AVG, MIN, MAX, and percentiles (P50, P95, P99) for trace and span metrics across specified MLflow experiments to monitor LLM costs and usage trends.

Can I group MLflow trace metrics by time intervals for hourly trend analysis?▼

Yes, you can group MLflow trace metrics by dimensions such as trace name or status and bucket them by time interval to perform hourly or daily trend analysis on LLM cost and quality.

What do I need to query trace metrics from an MLflow tracking server?▼

To query trace metrics from an MLflow tracking server, you need the server URL, experiment IDs, target metric, and desired aggregation. Optional time window and dimension parameters are supported for filtering.

Does MLflow metrics aggregation work for monitoring LLM cost spikes over a specific time window?▼

Yes, MLflow metrics aggregation works for monitoring LLM cost spikes by summarizing token consumption and latency over a specified time window, such as the last 24 hours, to identify slow endpoints.

Related Skills

Explore More Data & Analytics Skills→
ScottHMcKeanScottHMcKean

querying-mlflow-metrics

Fetch aggregated trace metrics from MLflow tracking servers.

Community
Intermediate
ice-droid-99ice-droid-99

querying-mlflow-metrics

Fetch aggregated MLflow trace metrics for token usage, latency, and quality scores.

Community
Intermediate
JingyiWngJingyiWng

querying-mlflow-metrics

Fetch aggregated MLflow trace metrics for token counts, latency, and assessment scores.

Community
Intermediate
byaan-aibyaan-ai
94

byaan:learn

Extract database models, tables, and schemas to generate accurate SQL queries.

Official
Advanced