omni-ai-eval

You can benchmark AI query generation accuracy by executing predefined eval prompts via the Omni CLI, capturing the generated query JSON, and scoring it against expected results to produce a clear accuracy metric.

Run regression tests for AI-generated query structures by executing standardized eval prompts to validate the topic, fields, filters, and sorts against expected results across different branches.

AI evaluation metrics for query generation scoring focus on validating specific query structures, specifically evaluating the accuracy of topic, fields, filters, and sorts within the generated JSON output.

Yes, you can compare model evaluation results across different branch configurations by running standardized eval sets through the Omni CLI and exporting the scored results to guide improvements.

Yes, the Omni CLI is required to execute model evaluation tasks, as the skill captures generated query JSON and scores it by enforcing evaluation tasks through standardized eval sets.

The best way to export AI evaluation results is by running eval prompts to generate exportable scored outputs, which can then be leveraged with tools like omni-ai-optimizer to drive reproducible improvements.