Evaluate and optimize LLM agent outputs using MLflow APIs.
npx skills add https://github.com/teegin-g/Slopcast --skill agent-evaluation-teegin-g
Please help me install this Agent Skill. Skill: agent-evaluation Source: https://github.com/teegin-g/Slopcast/tree/main/.agents/skills/agent-evaluation Command: npx skills add https://github.com/teegin-g/Slopcast --skill agent-evaluation-teegin-g
Evaluates and improves the performance and reliability of LLM agents by applying MLflow-based evaluation, tracing, and scoring to identify weaknesses, optimize tool selection, reduce costs, and fix inaccuracies in agent outputs.
Run an initial evaluation by preparing an MLflow-enabled environment, loading the agent, and following the four-step workflow to understand, configure scorers, prepare datasets, and run evaluation.
High-intent search queries and answers about installing and using this skill.
You can evaluate LLM agent outputs using MLflow by applying tracing setup, dataset creation, scorer definition, and execution scripts to identify weaknesses, optimize tool selection, and fix inaccuracies in agent workflows.
MLflow tracing for GenAI agents requires loading the instrumenting-with-mlflow-tracing skill first to integrate tracing APIs, which then enables the evaluation scripts to capture and measure end-to-end agent performance.
You can create datasets and define scorers for LLM evaluation by utilizing the provided templates and scripts in the scripts directory, which standardize the configuration of MLflow evaluation pipelines for component-level tasks.
Yes, MLflow evaluation workflows apply to both end-to-end agent workflows and individual components, providing step-by-step guidance for tracing setup, dataset preparation, scorer configuration, and evaluation execution.
MLflow agent evaluation requires access to MLflow APIs, an MLflow-enabled environment, and prior loading of the instrumenting-with-mlflow-tracing skill to successfully execute the four-step evaluation workflow.
MLflow scorers provide standardized evaluation pipelines that leverage tracing integration to identify agent weaknesses, reduce operational costs, and fix output inaccuracies across both full workflows and component-level tasks.
JingyiWngOrchestrate end-to-end GenAI agent evaluation with MLflow datasets, scorers, and tracing.
databricks-solutionsAutomate GenAI evaluation of ML agents with MLflow workflows.
aaronachermannEvaluate GenAI agents with MLflow datasets, scorers, and tracing.
m19cEvaluate LLM agent output quality using MLflow native APIs.
