skills.rest
mikeyobrien
mikeyobrienCOMMUNITY
Explore all skills by this author →
Website𝕏@mikeyobrienvGitHub

evaluate-presets

Evaluate Ralph's hat presets and generate logs, metrics, and summary reports.

3.1k|286|Updated Sep 7, 2025
Search Tags:#automation#testing#logs#metrics#CLI#presets#evaluate
One-click install
npx skills add https://github.com/mikeyobrien/ralph-orchestrator --skill evaluate-presets
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluate-presets
Source: https://github.com/mikeyobrien/ralph-orchestrator/tree/main/.claude/skills/evaluate-presets
Command: npx skills add https://github.com/mikeyobrien/ralph-orchestrator --skill evaluate-presets

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill automates the testing of Ralph's hat presets to ensure stability and UX quality.

Core Features & Use Cases

  • Single preset evaluation: Run and validate a specific preset against a backend.
  • Full-suite evaluation: Execute the entire preset library and generate consolidated metrics and logs.
  • Observability & QA aids: Produce logs, session data, and a summary report to guide fixes.

Quick Start

Use the evaluate-presets capability to run evaluations from the command line.

  • Run a single preset: ./tools/evaluate-preset.sh tdd-red-green claude
  • Evaluate all presets: ./tools/evaluate-all-presets.sh claude
  • Review results in the .eval directory: logs and results under .eval/logs and .eval/results

Frequently Asked Questions about evaluate-presets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate testing for hat presets from the command line?▼

You can automate hat preset testing by running the evaluate-presets CLI scripts, which validate individual presets or the full library against a backend and generate logs, metrics, and summary reports for QA.

Can I run a validation script on a single preset instead of the entire library?▼

Yes, you can run a single preset validation using the evaluate-preset.sh script with the preset name and backend as arguments, producing targeted logs and results without executing the full suite.

What logs and metrics are generated when evaluating the full preset library?▼

Evaluating the full preset library generates consolidated metrics, session data, and a summary report stored under the .eval directory, specifically within .eval/logs and .eval/results for QA analysis.

Do I need Bash tooling to run preset evaluation scripts?▼

Yes, Bash tooling is required to run the preset evaluation scripts and generate reproducible results, while optional YAML helpers provide additional configuration support during the testing process.

What is the best way to validate preset stability and UX quality before deployment?▼

The best way to validate preset stability is by executing the full-suite evaluation script, which runs the entire preset library and produces consolidated metrics and a summary report to guide necessary fixes.

Related Skills

Explore More Software Engineering Skills→
1k-off1k-off
1

package-script-writer

Generate and execute Umbraco CMS installation scripts via the psw CLI.

Community
Basic
NarubyRiverlioneNarubyRiverlione

task-completion-report

Parse tasks.md to compute per-phase and overall completion percentages.

Community
Basic
giosuetedeschi-specgiosuetedeschi-spec

langfuse

Track LLM traces, spans, and metrics for production AI applications.

Community
Advanced
0xtaoist0xtaoist

ln-782-test-runner

Detect and run test suites across Vitest, Jest, pytest, and more.

Community
Advanced