databricks-mlflow-evaluation

To evaluate GenAI agents using MLflow, you execute the `mlflow.genai.evaluate()` function with custom scorers and datasets. This process allows you to systematically measure agent quality, identify regressions, and analyze detailed execution traces for debugging.

Trace analysis for LLM agents involves examining detailed execution traces captured during evaluation to understand internal behavior. It helps developers debug performance issues by revealing exactly where failures occur within complex generative AI workflows.

Yes, you can automatically optimize prompts for LLMs using MLflow by leveraging GEPA with aligned judges. This feature systematically improves prompt quality by iteratively refining instructions based on evaluation feedback.

You can set up continuous production monitoring for live GenAI traffic by configuring an evaluation pipeline using MLflow. This approach continuously assesses live responses using built-in checks like Safety and Correctness to maintain quality standards.

Yes, MLflow evaluation supports custom scorers and advanced features like judge alignment with MemAlign for domain-specific quality assessment. You can define custom guidelines to ensure responses adhere to specific brand or retrieval requirements.

The best way to ensure RAG agent responses are grounded is to run `mlflow.genai.evaluate()` with the `RetrievalGroundedness` scorer. This validates that outputs remain factually tied to your retrieved documents and adhere to guidelines.