databricks-mlflow-evaluation

What problem does it solve?

MLflow GenAI evaluation workflows in Databricks environments are often ad-hoc and hard to reproduce; this Skill provides a structured, reference-backed approach to validate prompts, scorers, and domain-aligned judges across the evaluation lifecycle.

Core Features & Use Cases

• End-to-end evaluation patterns covering dataset creation, scorer configuration, trace analysis, and production monitoring for MLflow GenAI workloads.
• Support for the full domain-expert optimization loop: evaluation, labeling, judge alignment (MemAlign), automated prompt optimization (GEPA), and conditional production promotion.
• Built-in references and reference files (GOTCHAS, CRITICAL-interfaces, and patterns) to guide implementation, auditing, and repeatability.

Quick Start

Create a small eval dataset and run mlflow.genai.evaluate with your agent to observe baseline metrics and judge alignment.