skills.rest
google
googleOFFICIAL
Explore all skills by this author →
Website𝕏@GoogleOSSGitHub

agent-platform-eval-flywheel

Evaluate AI models and agents on Google Cloud using the GenAI Evaluation SDK.

17.1k|1.4k|Updated Mar 31, 2026
Search Tags:#model evaluation#ai evaluation#quality improvement#genai#agent platform
One-click install
npx skills add https://github.com/google/skills --skill agent-platform-eval-flywheel-google
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agent-platform-eval-flywheel
Source: https://github.com/google/skills/tree/main/skills/cloud/agent-platform-eval-flywheel
Command: npx skills add https://github.com/google/skills --skill agent-platform-eval-flywheel-google

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires google-cloud-aiplatform[evaluation], google-genai, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill enables you to measure and enhance the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. It helps you evaluate agents or models, create evaluation datasets, select evaluation metrics, analyze failures, compare results, and provides guidance on Agent Platform eval methodology.

Core Features & Use Cases

  • Evaluate AI Models and Agents: Assess the performance of AI models and agents using the Agent Platform GenAI Evaluation SDK.
  • Create Evaluation Datasets: Generate datasets from session traces, pandas DataFrames, or synthetic generation.
  • Select and Configure Metrics: Pick or write custom evaluation metrics and configure them for your specific needs.
  • Analyze Failures: Understand the root causes of failures and suggest concrete improvements.
  • Optimize & Iterate: Apply fixes and re-run evaluations to improve the performance of AI models and agents.

Quick Start

Install the required SDK:

pip install google-cloud-aiplatform[evaluation]>=1.154.0 google-genai>=1.0.0

Then, use the skill to evaluate a model:

python eval.py --model gemini-2.5-flash --dataset data.jsonl

Frequently Asked Questions about agent-platform-eval-flywheel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate the quality of AI agents on Google Cloud?▼

Evaluate AI agents on Google Cloud using the Agent Platform GenAI Evaluation SDK to measure performance, analyze failures, and iteratively improve quality.

How do I create evaluation datasets for GenAI models from session traces?▼

Create evaluation datasets for GenAI models by generating them directly from session traces, converting pandas DataFrames, or using synthetic generation methods.

Do I need the google-cloud-aiplatform SDK to run model evaluations?▼

Yes, you must install google-cloud-aiplatform[evaluation] and google-genai SDKs to execute model evaluations and apply the Eval Quality Flywheel methodology.

How can I analyze AI model failures and suggest improvements?▼

Analyze AI model failures by evaluating results against selected metrics, identifying root causes of errors, and applying suggested fixes before re-running evaluations to optimize performance.

Can I write custom evaluation metrics for my AI agents?▼

Yes, you can pick existing evaluation metrics or write custom metrics and configure them to assess the specific performance needs of your AI agents and models.

What is the best way to iteratively improve GenAI model performance?▼

Iteratively improve GenAI model performance by applying fixes to identified failures and re-running evaluations to compare results and optimize quality through the eval flywheel.

Related Skills

Explore More Software Engineering Skills→
wangx7wangx7

agent-platform-eval-flywheel

Evaluate and improve GenAI models and agents on Google Cloud.

Community
Advanced
Crispy13Crispy13

skill-creator

Automates the creation and iterative improvement of Skill documentation and evaluations.

Community
Advanced
ssmleossmleo

agent-platform-eval-flywheel

Evaluate AI models and agents using the Agent Platform GenAI Evaluation SDK.

Community
Advanced
sencersoylusencersoylu

ml-evaluation

Compute and summarize machine learning evaluation metrics for classifiers and regressors.

Community
Intermediate