skills.rest
ssmleo
ssmleoCOMMUNITY
Explore all skills by this author →
GitHub

agent-platform-eval-flywheel

Evaluate AI models and agents using the Agent Platform GenAI Evaluation SDK.

Updated Jul 4, 2026
Search Tags:#model evaluation#ai evaluation#quality assessment#genai#agent platform
One-click install
npx skills add https://github.com/ssmleo/govfolio --skill agent-platform-eval-flywheel-ssmleo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agent-platform-eval-flywheel
Source: https://github.com/ssmleo/govfolio/tree/main/.agents/skills/agent-platform-eval-flywheel
Command: npx skills add https://github.com/ssmleo/govfolio --skill agent-platform-eval-flywheel-ssmleo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires google-cloud-aiplatform[evaluation], google-genai, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive framework for evaluating and improving AI models and agents on Google Cloud using the Agent Platform GenAI Evaluation SDK.

Core Features & Use Cases

  • Evaluation Methodology: Utilizes the Eval Quality Flywheel methodology to measure and improve AI models and agents.
  • Dataset Creation: Builds and processes evaluation datasets from session traces, pandas DataFrames, or synthetic generation.
  • Metric Configuration: Allows selection, configuration, or writing custom evaluation metrics.
  • Failure Analysis: Diagnoses and suggests improvements based on rubric verdicts, loss patterns, and clustering failures.
  • Iterative Optimization: Supports iterative testing and optimization of AI models and agents.

Quick Start

Evaluate an AI model with the Agent Platform GenAI Evaluation SDK using the following command:

agent-platform-eval-flywheel evaluate my_model.json --metrics multi_turn_task_success, multi_turn_trajectory_quality

Frequently Asked Questions about agent-platform-eval-flywheel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate AI agents and models using Google Cloud?▼

To evaluate AI agents and models on Google Cloud, you can use the Agent Platform GenAI Evaluation SDK to run quality assessments, process datasets, and analyze multi-turn task success metrics.

How do I create evaluation datasets from session traces for GenAI?▼

You can create GenAI evaluation datasets by building and processing data directly from session traces, utilizing pandas DataFrames, or generating synthetic data within the evaluation workflow.

Does the GenAI evaluation SDK support custom evaluation metrics?▼

Yes, the GenAI evaluation SDK supports custom evaluation metrics, allowing you to select, configure, or write your own specific metrics to measure AI model quality accurately.

What is the best way to diagnose AI agent failures during quality assessment?▼

The best way to diagnose AI agent failures during quality assessment is by analyzing rubric verdicts, identifying loss patterns, and clustering failures to suggest targeted improvements.

Do I need Python client libraries to run model evaluation on Google Cloud?▼

Yes, you need the Python client libraries for Google Cloud AI Platform and GenAI installed in your environment to execute model evaluation and manage the iterative optimization process.

Related Skills

Explore More Product & Management Skills→
wangx7wangx7

agent-platform-eval-flywheel

Evaluate and improve GenAI models and agents on Google Cloud.

Community
Advanced
darkglow-netdarkglow-net
1

agentic-eval

Evaluate and refine AI agent outputs through iterative generate-evaluate-refine cycles.

Community
Advanced
googlegoogle
17.1k

agent-platform-eval-flywheel

Evaluate AI models and agents on Google Cloud using the GenAI Evaluation SDK.

Official
Advanced
leary-pokenleary-poken
1

databricks-mlflow-evaluation

Run MLflow GenAI evaluate() to score agent outputs with configurable scorers.

Community
Advanced