evaluate-presets

Automate preset testing by running Bash scripts like evaluate-preset.sh or evaluate-all-presets.sh from the command line. These tools execute standard tests, capture performance metrics, and output structured results to validate reliability and UX quality across updates.

Yes, preset regression checks support multiple backends. You can pass the backend name as the second argument to the evaluation scripts, such as specifying 'claude' or 'kiro', allowing you to compare current benchmark results against baselines and detect regressions across different environments.

The best way to evaluate all presets is by executing the evaluate-all-presets.sh script with the desired backend argument. This command-line tool runs automated tests across the entire preset library, captures metrics, and identifies any configuration issues to ensure they remain usable.

No, automated preset auditing relies solely on standard Bash scripts without meta-orchestration. You can run the tests directly from the command line by providing the preset name and backend, capturing structured metrics and validating UX quality without installing extra dependencies.

Validating preset UX quality during updates is necessary to identify configuration issues and ensure presets remain usable for developers. Automated auditing captures metrics to detect regressions early, preventing broken configurations from impacting the overall reliability of the hat system.

To test a single configuration, pass the preset name without the .yml extension as the first argument and the backend as the second. The backend argument accepts 'claude' or 'kiro' and defaults to 'claude' if omitted.