skills.rest
FMurray
FMurrayCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

querying-mlflow-metrics

Fetches aggregated trace metrics from MLflow tracking servers.

Updated Mar 5, 2026
Search Tags:#monitoring#llm evaluation#metrics#latency#token usage#mlflow#trace analytics
One-click install
npx skills add https://github.com/FMurray/mlfts --skill querying-mlflow-metrics-fmurray
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: querying-mlflow-metrics
Source: https://github.com/FMurray/mlfts/tree/main/.agents/skills/querying-mlflow-metrics
Command: npx skills add https://github.com/FMurray/mlfts --skill querying-mlflow-metrics-fmurray

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Manually inspecting individual MLflow traces makes it hard to understand aggregate behavior like token consumption, latency trends, error rates, and evaluation quality across experiments. This Skill queries the MLflow tracking server and returns aggregated metrics so you can analyze usage and quality at scale. ## Core Features & Use Cases - Aggregated Metrics: Compute COUNT, SUM, AVG, MIN, MAX, P50, P95, and P99 over trace_count, latency, and token usage metrics. - Time-Bucketed Trends: Group metrics into hourly or daily intervals with flexible start and end time ranges. - Dimensional Breakdowns: Group results by trace name, trace status, or assessment name, including LLM judge evaluation scores via the ASSESSMENTS view. - Use Case: An engineer wants to know how many tokens their coding agent consumed per hour over the last 24 hours and the average correctness score from LLM judges, then exports the results as JSON for a report. ## Quick Start Ask the agent to show the total and average token usage for experiment 1 on your MLflow server, optionally grouped by hour or trace status.

Frequently Asked Questions about querying-mlflow-metrics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query token usage metrics from MLflow?▼

Run the fetch_metrics.py script with the server URL, experiment IDs, and a token metric such as total_tokens, input_tokens, or output_tokens. Choose aggregations like SUM or AVG and optionally add a time interval to see hourly or daily trends.

How to analyze LLM evaluation scores stored in MLflow traces?▼

Use the ASSESSMENTS view with the assessment_value metric and group by assessment_name. This returns aggregate scores such as averages and medians for each evaluator, for example correctness or relevance judges.

What aggregations does the MLflow metrics query support?▼

Supported aggregations are COUNT, SUM, AVG, MIN, MAX, P50, P95, and P99. You can combine multiple aggregations in one query and group results by trace_name, trace_status, or assessment_name.

Can I filter MLflow metrics by time range?▼

Yes, use the start-time and end-time arguments with relative values like -24h or -7d, ISO 8601 timestamps, or epoch milliseconds. Add a time interval in seconds to bucket results, such as 3600 for hourly trends.

What are the limitations of querying MLflow trace metrics?▼

Queries require a reachable MLflow tracking server and valid experiment IDs, and only predefined metrics and dimensions are supported. For custom filters or complex joins, use the filter syntax described in the API reference.

Related Skills

Explore More Data & Analytics Skills→
BlackkadderBlackkadder

querying-mlflow-metrics

Fetch aggregated trace metrics from MLflow tracking servers.

Community
Intermediate
webrix-aiwebrix-ai
1

performance-monitoring

Set up performance monitoring with dashboards, alerts, and SLO tracking.

Official
Advanced
prashsubprashsub
1

mlflow-genai-evaluation

Automate Generative AI agent evaluation with MLflow GenAI and custom scorers.

Community
Advanced
choijinwonchoijinwon
1

querying-mlflow-metrics

Fetch and analyze MLflow metrics from tracking servers.

Community
Intermediate