databricks-mlflow-evaluation

Name: databricks-mlflow-evaluation
Availability: InStock
Author: zack2626-ok

Automates MLflow GenAI evaluation with built-in or custom scorers and GEPA prompt optimization.

Updated Jun 11, 2026

Search Tags:#prompt optimization #machine learning #evaluation #mlflow #genai

Ready to run

npx skills add https://github.com/Zack2626-ok/DATN_Website-Dat-Ban-Va-Quan-Ly-Nha-Hang --skill databricks-mlflow-evaluation-zack2626-ok

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-mlflow-evaluation
Source: https://github.com/Zack2626-ok/DATN_Website-Dat-Ban-Va-Quan-Ly-Nha-Hang/tree/main/.windsurf/skills/databricks-mlflow-evaluation
Command: npx skills add https://github.com/Zack2626-ok/DATN_Website-Dat-Ban-Va-Quan-Ly-Nha-Hang --skill databricks-mlflow-evaluation-zack2626-ok

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mlflow[databricks], openai, databricks, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the evaluation and optimization of MLflow GenAI models, saving time and reducing manual effort in the model development process.

Core Features & Use Cases

Automated Evaluation: Evaluate MLflow GenAI models with built-in scorers or custom scorers.
Prompt Optimization: Optimize prompts using GEPA for automated prompt improvement.
Trace Ingestion: Store and analyze production traces in Unity Catalog for continuous monitoring.
Judge Alignment: Align judges with domain expert feedback for improved evaluation accuracy.
Use Case: Imagine you have a GenAI model that needs evaluation and optimization. Use this Skill to automatically evaluate the model, optimize the prompts, and align judges for better results.

Quick Start

Use the databricks-mlflow-evaluation skill to evaluate a GenAI model and optimize the prompts.

Frequently Asked Questions about databricks-mlflow-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate MLflow GenAI evaluation and prompt optimization?▼

Automate MLflow GenAI evaluation and prompt optimization by using GEPA for automated prompt improvement alongside built-in or custom scorers. This Skill orchestrates the entire workflow to reduce manual model evaluation effort.

What is GEPA for automated prompt improvement in machine learning?▼

GEPA is an automated prompt improvement mechanism integrated into MLflow GenAI workflows. It systematically refines and optimizes prompts during the evaluation process to enhance model outputs without manual tuning.

Do I need Databricks and Unity Catalog to ingest production traces for MLflow monitoring?▼

Yes, storing and analyzing production traces for continuous monitoring requires Databricks and Unity Catalog. You also need MLflow version 3.1.0 or higher and the OpenAI library installed to support the trace ingestion workflow.

Can I align GenAI judges with domain expert feedback using MLflow?▼

Yes, you can align GenAI judges with domain expert feedback using the MemAlign mechanism. This improves evaluation accuracy by calibrating automated judges to closely reflect human expert assessments during model evaluation workflows.

What's the best way to evaluate MLflow GenAI models with custom scorers?▼

The best way to evaluate MLflow GenAI models with custom scorers is to run an automated evaluation workflow that integrates custom scoring logic with continuous production trace monitoring. This ensures accurate, context-specific model assessment.