skills.rest
broomva
broomvaCOMMUNITY
Explore all skills by this author →
Website𝕏@broomva_techGitHub

harness-selftest

Validate evaluation fixture integrity within CI/CD pipelines.

3|2|Updated Mar 17, 2026
Search Tags:#testing#evaluation#fixtures#ci#harness
One-click install
npx skills add https://github.com/broomva/skills --skill harness-selftest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: harness-selftest
Source: https://github.com/broomva/skills/tree/main/tests/skill_evals/fixtures/harness-selftest/skill
Command: npx skills add https://github.com/broomva/skills --skill harness-selftest

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill provides a controlled artifact for the evaluation harness to bind committed replay fixtures, ensuring that CI pipelines can verify prompt processing and grading logic without relying on external production tools.

Core Features & Use Cases

  • Evaluation Binding: Serves as a stable target for SHA-256 validation of test cases.
  • CI Gate Verification: Allows the evaluation harness to trigger and grade specific prompt sets to confirm that the testing infrastructure is functioning correctly.
  • Use Case: Developers use this to verify that the CI system correctly identifies and fails on modified prompt sets or invalid test fixtures.

Quick Start

Run the harness-selftest skill to validate the current evaluation fixture integrity.

Frequently Asked Questions about harness-selftest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate automated evaluation fixtures in a CI pipeline?▼

A CI gate verification uses a stable target to trigger and grade specific prompt sets. This confirms the testing infrastructure correctly identifies and fails on modified prompt sets or invalid test fixtures.

What is a deterministic artifact for binding replay fixtures?▼

A deterministic artifact for binding replay fixtures is a controlled target used by an evaluation harness. It ensures CI pipelines can verify prompt processing and grading logic through SHA-256 validation of test cases.

Why does my CI pipeline fail when validating modified prompt sets?▼

CI pipelines fail when validating modified prompt sets to confirm the testing infrastructure is functioning correctly. This behavior verifies the evaluation harness properly triggers and grades prompt-response cycles against committed replay fixtures.

Can I use a non-functional target for regression analysis of prompt-response cycles?▼

Yes, you can use a non-functional target for regression analysis of prompt-response cycles. It acts as a stable artifact for the evaluation harness, ensuring CI pipelines can verify prompt processing and grading logic deterministically.

What are the limitations of using a test-only fixture for CI gate verification?▼

The limitation of a test-only fixture is that it requires the evaluation harness to be specifically configured to recognize it. It is a non-functional target solely for regression analysis, not for executing actual production tools.

Related Skills

Explore More Software Engineering Skills→
zinohomezinohome
1

integration-test-builder

Generate integration tests for API endpoints with database seeding and CI strategies.

Community
Advanced
sangmokangsangmokang

vcx-dod-gate

Automate Phase 1 DoD gate verification with fresh build, lint, and test evidence.

Community
Advanced
ingo-eichhorstingo-eichhorst
93

ir:onboard-agent

Generate the canonical scenario×adapter fixture matrix for irrlicht with replay verification.

Community
Advanced
majunbaomajunbao

openai-whisper

Transcribe local audio files to text using the Whisper CLI.

Community
Intermediate