Evaluate LLM agent output quality using MLflow native APIs.
npx skills add https://github.com/m19c/dotfiles --skill agent-evaluation-m19c
Please help me install this Agent Skill. Skill: agent-evaluation Source: https://github.com/m19c/dotfiles/tree/main/claude/.claude/skills/agent-evaluation Command: npx skills add https://github.com/m19c/dotfiles --skill agent-evaluation-m19c
This Skill addresses the challenge of evaluating and optimizing the output quality of existing LLM agents, including improving tool selection accuracy, answer quality, and reducing costs.
Run the agent evaluation with MLflow by executing: uv run python scripts/evaluate_agent.py
High-intent search queries and answers about installing and using this skill.
You can evaluate LLM agent output quality using MLflow by leveraging its native APIs to systematically run evaluations with datasets, scorers, and tracing.
You can create custom scorers for evaluating LLM agents to measure specific quality criteria, such as tool selection accuracy and answer quality, within the evaluation workflow.
Yes, evaluating LLM agents requires MLflow and the instrumenting-with-mlflow-tracing skill to properly set up tracing for comprehensive observability and tracking during evaluation.
The best way to optimize LLM agents and reduce costs is through systematic evaluation using datasets and custom scorers to improve tool selection accuracy and overall answer quality.
Yes, you can run a step-by-step dry run to catch issues early and validate your datasets and scorers before executing the full LLM agent evaluation.
ice-droid-99Evaluate and improve LLM agent outputs using MLflow GenAI workflows.
aaronachermannEvaluate GenAI agents with MLflow datasets, scorers, and tracing.
JingyiWngOrchestrate end-to-end GenAI agent evaluation with MLflow datasets, scorers, and tracing.
DhumitechAutomate AI agent evaluation and benchmarking with repeatable test pipelines.
