mlflow-genai-evaluation

Evaluate GenAI agents using MLflow by implementing evaluation pipelines with the `mlflow.genai.evaluate()` function, utilizing LLM-as-judge scorers for nuanced quality assessment before deployment.

Create custom scorers for LLM evaluation by developing domain-specific metrics and integrating them seamlessly using the `_extract_response_text()` helper to parse and evaluate agent responses.

Align LLM judges with domain feedback by applying the MemAlign judge alignment patterns, ensuring evaluation metrics accurately reflect specific quality and safety standards.

Optimize evaluation prompts for GenAI agents by applying GEPA prompt optimization techniques, which refine scorer instructions to improve evaluation accuracy and threshold checking.

Set up production monitoring for Databricks GenAI agents by configuring Unity Catalog trace ingestion, enabling continuous performance tracking and automated threshold checking for production readiness.

MLflow GenAI evaluation works with the Databricks SDK by utilizing it for underlying LLM calls, allowing you to execute evaluation pipelines and metric aliases within the Databricks environment.