databricks-mlflow-evaluation

GenAI evaluation workflows in MLflow require running mlflow.genai.evaluate() with a local predict function and an evaluation dataset to generate initial results. The Skill provides workflow templates and best-practice references to guide setup, trace analysis, and production monitoring loops.

Production monitoring for GenAI agents involves continuously tracking agent behavior through trace ingestion. This Skill provides patterns for ingesting Unity Catalog traces to analyze agent performance, monitor quality, and maintain robust evaluation experiments over time.

MLflow evaluation supports judge alignment and prompt optimization through MemAlign-based patterns and GEPA-inspired techniques. These patterns integrate datasets, traces, and scorers to refine evaluation quality and improve GenAI agent outputs during experiments.

MLflow GenAI evaluation explicitly supports custom scorer integration and evaluation datasets. Teams can configure robust experiments by combining their own datasets and scorer logic with MLflow's end-to-end evaluation loops and trace analysis features.

Trace analysis is needed to inspect GenAI agent execution paths and identify quality issues. This Skill provides reference files and best-practice documentation to help teams parse traces, optimize prompts, and establish reliable production monitoring workflows.