skills.rest
juanlamadrid20
juanlamadrid20COMMUNITY
Explore all skills by this author →
GitHub

mlflow-evaluation

Automate MLflow GenAI agent evaluation across traces, datasets, and scorers.

Updated Sep 29, 2025
Search Tags:#evaluation#latency#traces#scorers#mlflow#datasets#genai
One-click install
npx skills add https://github.com/juanlamadrid20/dbrx-multi-agent-retail-intelligence --skill mlflow-evaluation-juanlamadrid20
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mlflow-evaluation
Source: https://github.com/juanlamadrid20/dbrx-multi-agent-retail-intelligence/tree/main/.claude/skills/mlflow-evaluation
Command: npx skills add https://github.com/juanlamadrid20/dbrx-multi-agent-retail-intelligence --skill mlflow-evaluation-juanlamadrid20

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

MLflow GenAI Evaluation helps teams systematically assess and compare the quality, safety, and relevance of GenAI agent behaviors using MLflow 3 GenAI evaluation workflows.

Core Features & Use Cases

  • Define evaluation datasets from traces, production logs, or in-memory records.
  • Configure built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness) and custom evaluators.
  • Run end-to-end evaluations with mlflow.genai.evaluate(), analyze traces for latency, errors, and architecture, and compare iterations over time.

Quick Start

To start evaluating your agent, prepare a small eval dataset, implement a simple scorer, and execute mlflow.genai.evaluate() against your locally imported agent.

Frequently Asked Questions about mlflow-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate GenAI agent traces using MLflow?▼

To evaluate GenAI agent traces using MLflow, you can create evaluation datasets from traces, configure built-in scorers like Guidelines or Correctness, and run mlflow.genai.evaluate() to analyze trace latency, errors, and architecture.

What built-in scorers are available for MLflow GenAI evaluation?▼

MLflow GenAI evaluation supports built-in scorers including Guidelines, Correctness, Safety, and RetrievalGroundedness. You can configure these to assess agent quality, safety, and relevance across your evaluation datasets.

Can I use production logs to create evaluation datasets in MLflow?▼

Yes, you can create evaluation datasets from production logs, traces, or in-memory records to systematically assess and compare GenAI agent behaviors using MLflow 3 evaluation workflows.

How do I analyze agent latency and errors in MLflow traces?▼

You can analyze agent latency and errors by running end-to-end evaluations with mlflow.genai.evaluate(), which enables you to inspect trace architecture and per-stage metrics to compare iterations over time.

Does MLflow GenAI evaluation work with Databricks tooling?▼

Yes, MLflow GenAI evaluation supports integration with MLflow and Databricks tooling, allowing you to run evaluations, analyze traces, and compare agent iterations within your existing data analytics environment.

Related Skills

Explore More Data & Analytics Skills→
JingyiWngJingyiWng

agent-evaluation

Orchestrate end-to-end GenAI agent evaluation with MLflow datasets, scorers, and tracing.

Community
Advanced
DhumitechDhumitech

agent-evaluation

Automate AI agent evaluation and benchmarking with repeatable test pipelines.

Community
Advanced
aaronachermannaaronachermann

databricks-mlflow-evaluation

Evaluate GenAI agent outputs with MLflow GenAI evaluation APIs.

Community
Advanced
BlackkadderBlackkadder

agent-evaluation

Evaluate and optimize LLM agent output quality using MLflow evaluation APIs.

Community
Advanced