mlflow-evaluation

MLflow GenAI evaluation enables safety, correctness, relevance, and grounding checks by orchestrating traces, evaluation datasets, scorers, and reference guidance to produce repeatable assessment runs across multiple dimensions.

You need a Python environment with MLflow installed, prepared evaluation data, a local predict_fn wrapper for your agent, and configured scorers to execute end-to-end GenAI evaluation runs successfully.

Yes, MLflow GenAI evaluation supports optional Databricks integrations. You can install MLflow with Databricks extras to enable connected tracking and evaluation runs within your Databricks environment.

Benchmarking with mlflow.genai.evaluate works by running standardized evaluation runs across agent versions, using consistent datasets and scorers to compare results for CI/CD, experimentation, and benchmarking.

The best way to compare agent performance is using mlflow.genai.evaluate to run standardized evaluation runs with consistent datasets and scorers, enabling cross-version comparisons for CI/CD and benchmarking.

MLflow GenAI evaluation workflows require a pre-configured Python environment with MLflow installed and properly prepared evaluation datasets, limiting quick ad-hoc usage without upfront environment and data preparation.