skills.rest
prince3626ezechiel-lang
prince3626ezechiel-langCOMMUNITY
Explore all skills by this author →
GitHub

caliper-skill-eval

Execute Caliper skills and measure success rates for pre-integration testing.

Updated Jul 5, 2026
Search Tags:#integration testing#skill testing#end-to-end testing#skill evaluation#Caliper
One-click install
npx skills add https://github.com/prince3626ezechiel-lang/ivoire-monade-palantir --skill caliper-skill-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: caliper-skill-eval
Source: https://github.com/prince3626ezechiel-lang/ivoire-monade-palantir/tree/main/skills/caliper-skill-eval
Command: npx skills add https://github.com/prince3626ezechiel-lang/ivoire-monade-palantir --skill caliper-skill-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires caliper, and includes scripts (resource) components.

What problem does it solve?

This Skill provides a lightweight evaluation wrapper for Caliper, ensuring that Hermes/Codex/Pi skills work end-to-end before integration.

Core Features & Use Cases

  • Skill Evaluation: Verify the functionality of skills by executing them and measuring the success rate.
  • Pre-integration Testing: Use before adding skills to local repositories or relay runs to prevent regressions.
  • Scheduled Reviews: Automate checks during scheduled tasks or PR reviews to ensure continuous integration.

Quick Start

Run the caliper-skill-eval skill to evaluate the 'example-skill' and check its pass rate.

Frequently Asked Questions about caliper-skill-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run end-to-end testing for Caliper skills before integration?▼

Prevent regressions in Hermes, Codex, or Pi skills by running an evaluation wrapper that measures their execution success rate. This validates end-to-end functionality before you add skills to local repositories or relay runs.

Can I automate skill evaluation during scheduled tasks or PR reviews?▼

Yes, you can automate skill evaluation during scheduled tasks or PR reviews. This continuous integration check measures execution success rates to ensure Hermes, Codex, or Pi skills maintain functionality without manual intervention.

What is skill evaluation and how does it prevent regressions?▼

Skill evaluation is the process of executing a skill and measuring its success rate to verify end-to-end functionality. By testing skills before integration, it identifies failures early and prevents regressions in your repository.

Do I need the Caliper library to run integration testing on skills?▼

Yes, the Caliper library is required as a dependency for skill execution and result analysis. The evaluation scripts rely on Caliper to run the end-to-end tests and measure the success rates of your skills.

What are the limitations of using a lightweight evaluation wrapper for skill testing?▼

As a lightweight evaluation wrapper, this skill focuses on measuring execution success rates for pre-integration testing. It is designed for scheduled tasks and PR reviews, meaning it may lack deep diagnostic capabilities for complex skill failures.

Related Skills

Explore More Software Engineering Skills→
aviz85aviz85
45

aviz-skills-installer

Install and manage Claude Code skills from the AVIZ Skills Library.

Community
Intermediate
polsegupolsegu

test-flakiness

Detect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.

Community
Intermediate
coval-aicoval-ai
2

onboard

Configure and execute AI evaluation workflows on the Coval platform.

Official
Advanced
zardusai-cyberzardusai-cyber
1

canary-watch

Monitor deployed URLs for regressions and health issues after deployments.

Community
Intermediate