databricks-mlflow-evaluation

Run MLflow GenAI evaluation by calling mlflow.genai.evaluate() with an eval dataset containing nested inputs, optional outputs or expectations, and the appropriate built-in scorers for correctness, safety, and retrieval grounding.

Yes, you can convert production traces into repeatable evaluation datasets. The Skill provides guidance for trace to dataset conversion, enabling you to turn staged or production runs into structured records for continuous MLflow GenAI evaluation.

Judge alignment ensures domain-accurate quality signals by using MemAlign to align scorers with your specific evaluation criteria, improving the reliability of automated MLflow GenAI evaluations over time.

Automate prompt optimization using GEPA's optimize_prompts() function. The Skill outlines the prerequisites for GEPA prompt optimization, allowing you to systematically improve agent response quality based on evaluation results.

Yes, the workflow supports ingesting production traces into Unity Catalog. It provides specific guidance for Unity Catalog trace storage and continuous scoring to monitor GenAI agent quality in production environments.

mlflow.genai.evaluate() requires specific predict_fn calling semantics because the evaluation engine expects properly structured records with nested inputs. Correct semantics ensure built-in scorers and judges receive the data format needed to measure response quality accurately.