examples-run-analysis

Analyzes completed example workflow logs and source files to validate run results.

3.7k|945|Updated May 31, 2025
One-click install
npx skills add https://github.com/openai/openai-agents-js --skill examples-run-analysis
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: examples-run-analysis
Source: https://github.com/openai/openai-agents-js/tree/main/.agents/skills/examples-run-analysis
Command: npx skills add https://github.com/openai/openai-agents-js --skill examples-run-analysis

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

After running the repository's example workflow, contributors need to verify that every example actually behaved correctly, not just exited with code zero. This Skill inspects the completed run's logs and source files to validate behavior, classify failures, and explain skips without ever executing or controlling any process.

Core Features & Use Cases

  • Behavioral validation: Compares each exit-zero example's log against its intended flow in source code, confirming expected tool calls, approvals, handoffs, streamed output, and generated artifacts.
  • Failure and skip classification: Categorizes non-zero exits as repository failures, expected example failures, external dependency restrictions, or environment limits, and classifies every skip by its recorded reason.
  • Freshness and completeness checks: Detects stale, incomplete, or still-running logs and refuses to analyze them, directing the user to finish the manual pnpm run first.
  • Use Case: After running pnpm examples:workflow:start, ask the Skill to review the latest logs; it reports one line per example with evidence, plus separate lists of failures, intentional skips, and environment restrictions.

Quick Start

Use the examples-run-analysis skill to inspect the latest completed example run logs and tell me which examples genuinely passed, which failed, and which were skipped.

Frequently Asked Questions about examples-run-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify that repository examples actually worked after a workflow run?

Inspect the newest completed main log in .tmp/examples-start-logs and compare each exit-zero example's log against its intended behavior in source files. Exit code zero alone is not behavioral validation; confirm expected tool calls, outputs, and artifacts occurred.

How to classify failed examples in a workflow log?

Classify each non-zero exit as a repository failure, an expected example-level failure, an external dependency or credential restriction, a host/environment restriction, or incomplete evidence. Quote the concise untruncated log lines that justify each classification.

Can this skill rerun or start the example workflow for me?

No. The skill is strictly read-only and never invokes pnpm, Node.js, builds, tests, or any workflow command. When evidence is missing or stale, it asks you to run pnpm examples:workflow:start manually and return after completion.

Why does the analysis refuse to use my latest log file?

The log is rejected when it lacks the complete footer with the final Done line and result table, indicates cancellation or an active process, cannot be reconciled with recorded starts and skips, or predates the run you asked about.

What do skipped examples in the run results mean?

Each skip is classified by its recorded reason: interactive, server, audio, external service, default auto-skip, conditional missing environment, or runner exclusion. The analysis distinguishes intentional skips from failures and environment restrictions.