skills.rest
JingyiWng
JingyiWngCOMMUNITY
Explore all skills by this author →
GitHub

querying-mlflow-metrics

Fetch aggregated MLflow trace metrics for token counts, latency, and assessment scores.

Updated Mar 25, 2026
Search Tags:#llm#metrics#aggregation#trace#assessment#cost#mlflow
One-click install
npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill querying-mlflow-metrics-jingyiwng
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: querying-mlflow-metrics
Source: https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher/tree/main/.claude/skills/querying-mlflow-metrics
Command: npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill querying-mlflow-metrics-jingyiwng

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

The skill removes the manual effort of querying MLflow tracking servers for aggregated metrics like token usage, latency, and quality assessments, enabling quick insight into LLM operational costs and performance.

Core Features & Use Cases

  • Retrieve aggregated token usage, latency, and trace counts across experiments.
  • Generate time‑bucketed trends (hourly, daily) for monitoring LLM cost and performance.
  • Summarize assessment scores from custom evaluators to evaluate response quality.

Quick Start

Ask the skill to report the average total tokens used in the last 24 hours for experiment 1.

Frequently Asked Questions about querying-mlflow-metrics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I retrieve aggregated token counts and latency from MLflow traces?▼

To retrieve aggregated MLflow trace metrics like token counts and latency, apply this tool to an MLflow tracking server to fetch and summarize usage data across experiments using customizable command-line arguments.

Can I group MLflow trace metrics by time intervals for monitoring LLM cost?▼

Yes, you can generate time-bucketed trends for monitoring LLM cost and performance by applying hourly or daily intervals to group your aggregated MLflow trace statistics and token usage.

What is the best way to summarize assessment scores from custom evaluators in MLflow?▼

The best way to summarize MLflow assessment scores from custom evaluators is to query the tracking server for aggregated response quality metrics grouped by your specified dimensions.

Does querying MLflow metrics require any external dependencies?▼

No external dependencies are required to query MLflow metrics. The skill operates independently to fetch trace statistics, token counts, latency, and assessment scores directly via command-line arguments.

How do I monitor LLM performance trends across multiple experiments in MLflow?▼

You can monitor LLM performance trends by fetching aggregated MLflow trace metrics across multiple experiments, applying customizable grouping dimensions and time-bucketed intervals to analyze usage and latency over time.

Related Skills

Explore More Data & Analytics Skills→
aaronachermannaaronachermann

querying-mlflow-metrics

Fetch aggregated MLflow trace metrics with time-range bucketing and JSON or table output.

Community
Intermediate
ice-droid-99ice-droid-99

querying-mlflow-metrics

Fetch aggregated MLflow trace metrics for token usage, latency, and quality scores.

Community
Intermediate
databrickslabsdatabrickslabs
38

databricks-mlflow-evaluation

Evaluate GenAI agents with MLflow 3 GenAI evaluation APIs and built-in scorers.

Official
Advanced
mirakuimirakui

querying-mlflow-metrics

Fetch aggregated trace metrics from MLflow tracking servers.

Community
Intermediate