run-e2e

Dispatch, monitor, and evaluate connections-checker E2E runs against main baselines on the AllSpice Hub.

Updated Apr 3, 2021
One-click install
npx skills add https://github.com/shrik450/dotfiles --skill run-e2e-shrik450
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: run-e2e
Source: https://github.com/shrik450/dotfiles/tree/main/dot_claude/skills/run-e2e
Command: npx skills add https://github.com/shrik450/dotfiles --skill run-e2e-shrik450

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires py-allspice, python-dotenv, and includes scripts (resource) components.

What problem does it solve? Running connections-checker end-to-end evaluations involves dispatching GitHub workflows, polling the genai-eval AllSpice Hub for results, and judging noisy metrics against baselines. This Skill encodes that entire procedure so a single E2E round is executed and interpreted correctly without relearning the pitfalls each time. ## Core Features & Use Cases - Dispatch and follow runs: Kick off e2e_manual.yml with a full 40-character SHA, then poll run status, design reviews, and comments via the hub.py helper script. - Summarize results: Use summarize.py to condense rubric judge comments and metrics-index bands into compact per-run blocks for comparison against main baselines. - Pull artifacts and traces: Download eval_output artifacts with pull_artifact.py to inspect rendered page input, model reasoning groups, and verifier confidence reports. - Use Case: After pushing a branch that changes how KiCad schematics are rendered, dispatch an E2E round, wait 15-25 minutes, then compare caught rubric items and token usage against four recent main baselines to confirm the change engaged without regressions. ## Quick Start Ask the assistant to run an E2E round for the current connections-checker branch and report how it compares to recent main baselines.

Frequently Asked Questions about run-e2e

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run connections-checker E2E evaluations on a branch?

Push the ref to GitHub, then dispatch the e2e_manual.yml workflow with the full 40-character SHA using gh workflow run. Poll run status with the hub.py tasks subcommand every 90 to 120 seconds until the design reviews complete.

How do I compare an E2E run against main baselines?

Find recent main runs with hub.py drs, taking four including the branch's merge base, then run summarize.py on each design review. Judge caught rubric items, false-positive buckets, and token usage against the baseline spread rather than a single run.

Why do all E2E repos fail within about 30 seconds?

An abbreviated commit SHA causes every run to fail in roughly 30 seconds because the injected workflow cannot resolve the action ref. Re-dispatch using the full 40-character SHA from git rev-parse HEAD.

Can I read E2E job logs through the AllSpice Hub API?

No. The Gitea 1.24 Hub exposes no job-log endpoint, and the web log route requires a session cookie rather than a token. When a run fails early, ask the user to paste the failing step from the run page.

How do I inspect what the model actually saw during an E2E run?

Download the eval_output artifact with pull_artifact.py and read debug_dump pages input.txt, which contains the exact rendered page text fed to the model. Diffing it against a baseline run shows precisely what the change introduced.