databricks-mlflow-evaluation

You can evaluate MLflow GenAI agents by calling mlflow.genai.evaluate with your agent and desired scorers. This built-in process measures agent performance, quality, and domain-specific standards like safety and correctness.

The best way to ingest production traces is through Unity Catalog trace ingestion. This enables continuous production monitoring and helps build datasets for evaluating MLflow GenAI agent performance over time.

Yes, MLflow GenAI evaluation supports custom scorer development and judge alignment. You can build datasets, run custom scorers, and align judges to meet domain-specific correctness and relevance requirements.

Automated prompt optimization in MLflow uses the GEPA method to refine GenAI agent prompts. This framework integrates with evaluation workflows to iteratively improve agent performance and quality.

Common gotchas include misconfiguring trace ingestion and misaligning judges. The framework provides integrated reference guides for critical interfaces and pattern libraries to help avoid these common pitfalls.

Yes, MLflow evaluation supports regression detection by running structured end-to-end workflows. You can monitor production traces and run scorers to detect performance regressions in GenAI agents.