skills.rest
teegin-g
teegin-gCOMMUNITY
Explore all skills by this author →
GitHub

databricks-mlflow-evaluation

Automate MLflow GenAI agent evaluation with trace data and reproducible workflows.

Updated Jan 30, 2026
Search Tags:#monitoring#evaluation#trace#alignment#mlflow#datasets#genai
One-click install
npx skills add https://github.com/teegin-g/Slopcast --skill databricks-mlflow-evaluation-teegin-g
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-mlflow-evaluation
Source: https://github.com/teegin-g/Slopcast/tree/main/.agents/skills/databricks-mlflow-evaluation
Command: npx skills add https://github.com/teegin-g/Slopcast --skill databricks-mlflow-evaluation-teegin-g

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Automates evaluation of MLflow GenAI agents using MLflow's GenAI APIs and trace data.

Core Features & Use Cases

  • End-to-end evaluation: datasets, scorers, alignments, and monitoring for GenAI agents.
  • Trace-driven quality: leverages production traces and MLflow traces to measure alignment and performance.
  • Production readiness: supports MemAlign alignment and GEPA-based prompt optimization workflows.

Quick Start

Analyze a sample agent evaluation by running a local evaluation script against a small eval_dataset.

Frequently Asked Questions about databricks-mlflow-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate MLflow GenAI agents using production trace data?▼

You can evaluate MLflow GenAI agents by automating workflows with mlflow.genai.evaluate() and analyzing production trace data to measure alignment and performance. This enforces reproducible workflows from data ingestion to production monitoring.

What is MemAlign alignment for GenAI agent evaluation?▼

MemAlign alignment is a production readiness technique used within MLflow GenAI agent evaluation workflows. It measures how well agent outputs align with expected behaviors to ensure production quality.

How do I optimize prompts using GEPA and MLflow traces?▼

You can optimize prompts by applying GEPA-based prompt optimization workflows to your MLflow trace analysis. This leverages production traces to drive automated prompt improvements for GenAI agents.

Can I build custom scorers and datasets for mlflow.genai.evaluate()?▼

Yes, you can create custom datasets and design specific scorers for mlflow.genai.evaluate(). The workflow supports end-to-end evaluation projects including data ingestion, dataset creation, and judge alignment.

Does this MLflow evaluation workflow support end-to-end production monitoring?▼

Yes, the MLflow evaluation workflow supports end-to-end production monitoring for GenAI agents. It leverages production traces and MLflow traces to continuously measure alignment and performance.

Related Skills

Explore More Data & Analytics Skills→
databricks-solutionsdatabricks-solutions
1.8k

mlflow-evaluation

Automate GenAI evaluation of ML agents with MLflow workflows.

Official
Advanced
JingyiWngJingyiWng

databricks-mlflow-evaluation

Evaluate MLflow GenAI agents with built-in and custom metrics.

Community
Advanced
terrylicaterrylica
62

mlflow-python

Log MLflow metrics, parameters, and artifacts using the Python API.

Community
Advanced
thbehthbeh

databricks-mlflow-evaluation

Evaluate MLflow 3 GenAI agents with automated workflows and Unity Catalog integration.

Community
Advanced