skills.rest
amd
amdOFFICIAL
Explore all skills by this author →
WebsiteGitHub

gaia-testing

Runs unit, integration, and real-world hardware test tiers for GAIA with screenshot and trace evidence.

1.6k|168|Updated Dec 16, 2024
Search Tags:#testing#screenshots#e2e#regression#playwright#gaia#eval
One-click install
npx skills add https://github.com/amd/gaia --skill gaia-testing-amd
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gaia-testing
Source: https://github.com/amd/gaia/tree/main/.claude/skills/gaia-testing
Command: npx skills add https://github.com/amd/gaia --skill gaia-testing-amd

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Green unit tests routinely miss shipped-but-broken features — a RAG feature that works in the backend but is blocked in the UI, or a release-note claim contradicted by the source. This Skill closes that gap by testing GAIA changes across unit, integration, and real-world tiers, driving the actual Agent UI, CLI, MCP, and API surfaces on real AMD Ryzen AI hardware and returning verifiable evidence instead of prose claims. ## Core Features & Use Cases - Multi-tier regression harness: Runs unit tests with pre-existing-failure isolation, integration tests through the real CLI, and a real-world tier that drives the live Agent UI via Playwright or the Agent UI MCP (gaia mcp serve). - Evidence-first verification: Captures screenshots, logs, traces, planted-fact retrieval proof, and per-operation timing, then embeds sanitized screenshots directly in PR descriptions via raw.githubusercontent.com evidence branches. - LLM-behaviour evals: Runs gaia eval agent against a live backend and diffs scorecards against committed baselines to catch regressions from prompt, tool, or model changes. - Use Case: Before merging a PR that changes the Chat agent's UI, the Skill plans the tiers, gets one approval, drives the Agent UI in a real browser on a Ryzen AI machine, injects an unguessable planted fact, and delivers captioned before/after screenshots plus a verdict table on the PR. ## Quick Start Ask the agent to test and validate a GAIA feature or release end-to-end on real hardware with screenshots as proof.

Frequently Asked Questions about gaia-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a GAIA feature end-to-end on real hardware?▼

Ask to test or validate the feature with real-world evidence. The Skill plans unit, integration, and real-world tiers, gets one approval, then drives the live Agent UI with Playwright on a Ryzen AI machine and returns screenshots, logs, and traces.

What evidence does a GAIA UI change need on a pull request?▼

An Agent UI screenshot is required for any change exposed in the Agent UI — API, CLI, or MCP text output does not substitute. Screenshots are embedded in the PR description as raw.githubusercontent.com images so they render inline.

How does gaia eval agent baseline comparison work?▼

Start the eval backend with python -m gaia.ui.server on port 4200, run gaia eval agent for the category, then use --compare with the committed baseline scorecard and the new scorecard. A material drop below baseline blocks the change.

Does the real-world tier run when no GPU or test machine is available?▼

No. Phase 0 pre-flights hardware capability and excludes impossible tiers up front with a stated reason, running only unit and integration tiers. A tier that fails mid-run stops with an error rather than degrading to a partial pass.

Why use planted facts when testing RAG or search features?▼

Planted unguessable values like a unique passphrase prove live retrieval instead of hallucinated or cached responses. The judge verifies the fact appears both in the UI screenshot and in the raw agent trace.

When should I use the verify skill instead of gaia-testing?▼

Use verify for a quick check that the app is running. Use gaia-testing when you need end-to-end proof that a feature, fix, or release actually works, with screenshots and multi-tier evidence.

Related Skills

Explore More Software Engineering Skills→
1t1sCooL1t1sCooL

aif-qa

Generates change summaries, test plans, and test cases from git branch diffs.

Community
Advanced
Zhachory1Zhachory1

pr-review

Reviews pull requests against intent and publishes verdict-mapped GitHub reviews with prioritized findings.

Community
Advanced
sizukutamagosizukutamago

verified-commit

Run lint, type-check, and tests before commits.

Community
Intermediate
samykabusamykabu

how-to-test

Generate QA-facing How-To-Test manuals from workspace configurations and feature artifacts.

Community
Advanced