skills.rest
lucaslessachaves
lucaslessachavesCOMMUNITY
Explore all skills by this author →
GitHub

databricks-mlflow-evaluation

Automate MLflow GenAI agent evaluation with scoring, MemAlign judge alignment, and GEPA prompt optimization.

1|Updated Dec 15, 2025
Search Tags:#dataset#mlflow#prompt-optimization#production-monitoring#gepa#memalign#genai-evaluation
One-click install
npx skills add https://github.com/lucaslessachaves/default --skill databricks-mlflow-evaluation-lucaslessachaves
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-mlflow-evaluation
Source: https://github.com/lucaslessachaves/default/tree/main/.claude/skills/databricks-mlflow-evaluation
Command: npx skills add https://github.com/lucaslessachaves/default --skill databricks-mlflow-evaluation-lucaslessachaves

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

MLflow GenAI evaluation and production-ready prompt optimization require coordinating evaluations, judge alignment, and monitoring across traces, datasets, and scoring configurations. This Skill provides a repeatable workflow to run evaluations, align domain-specific judges with MemAlign, and optimize prompts with GEPA for production readiness.

Core Features & Use Cases

  • End-to-end evaluation scaffolding for MLflow GenAI, including built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), dataset management, and production monitoring.
  • MemAlign-based judge alignment to embed domain-expert preferences into evaluation signals.
  • GEPA-based automated prompt optimization to improve prompts and promote production-ready versions.

Quick Start

Run an initial evaluation using mlflow.genai.evaluate() with a prepared eval dataset and the default base judge to generate baseline metrics.

Frequently Asked Questions about databricks-mlflow-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate MLflow GenAI agents using custom domain criteria?▼

To evaluate MLflow GenAI agents, use mlflow.genai.evaluate() with built-in scorers and MemAlign to embed domain-expert preferences into your evaluation signals for actionable metrics.

What's the best way to optimize prompts for production deployment in MLflow?▼

Optimize prompts for production deployment by applying GEPA-based automated prompt optimization to improve prompts and promote production-ready versions from your evaluation datasets.

How does MemAlign judge alignment work for GenAI evaluation datasets?▼

MemAlign judge alignment works by embedding domain-expert preferences into evaluation signals, coordinating judge alignment with built-in scorers like Guidelines and RetrievalGroundedness across traces.

Can I build evaluation datasets from production traces in MLflow?▼

Yes, you can build evaluation datasets from production traces by applying trace ingestion workflows, coordinating dataset management with production monitoring to generate baseline metrics.

Do I need prior knowledge of scoring patterns to use MLflow GenAI evaluation?▼

Yes, you need knowledge of mlflow.genai.evaluate(), scoring patterns, trace ingestion, and prompt optimization to produce actionable metrics and guidance from the evaluation workflow.

What built-in scorers are available for MLflow GenAI evaluation?▼

Built-in scorers for MLflow GenAI evaluation include Guidelines, Correctness, Safety, and RetrievalGroundedness, which coordinate with domain-aligned judges to score agent outputs.

Related Skills

Explore More Data & Analytics Skills→
databricks-solutionsdatabricks-solutions
1.8k

mlflow-evaluation

Automate GenAI evaluation of ML agents with MLflow workflows.

Official
Advanced
Aradhya0510Aradhya0510
3

databricks-mlflow-evaluation

Evaluate Generative AI agents with MLflow 3 GenAI APIs and custom scorers.

Community
Advanced
juanlamadrid20juanlamadrid20

mlflow-evaluation

Automate MLflow GenAI agent evaluation across traces, datasets, and scorers.

Community
Advanced
databrickslabsdatabrickslabs
38

databricks-mlflow-evaluation

Evaluate GenAI agents with MLflow 3 GenAI evaluation APIs and built-in scorers.

Official
Advanced