quality

Validates skill quality through falsification testing and golden test coverage methodology.

6|1|Updated May 11, 2026
One-click install
npx skills add https://github.com/yakeworld/Synthos --skill quality-yakeworld
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: quality
Source: https://github.com/yakeworld/Synthos/tree/main/skills/private/extended/research-tools/synthos-akne-bridge/quality
Command: npx skills add https://github.com/yakeworld/Synthos --skill quality-yakeworld

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Ensuring that skills actually work as claimed is hard: tests often use mock data, only check happy paths, or lack reproducible evidence. This Skill enforces rigorous quality validation by routing checks to two sub-skills — falsification-validation and golden-test-methodology — so every skill claim is tested against real inputs and golden test coverage is measured consistently. ## Core Features & Use Cases - Falsification Validation: Tests skill claims using real task inputs (never mock data), requires counter-evidence tests, archives full evidence chains to test-results/, and maintains a Bayesian trust score per skill via the update_trust.py script. - Golden Test Methodology: Computes golden test coverage (skills with complete GOLDEN_SET.md + cases/ + expected/), grades coverage (Excellent/Adequate/Low/Critical), and defines prioritized expansion order for missing tests. - Input Validation & Routing: Rejects empty or invalid skill_path inputs with contextual error messages and recovery suggestions, then routes valid requests to the appropriate sub-skill. - Use Case: When the evolution engine needs to verify a newly absorbed skill, invoke this Skill with the skill's path to run falsification tests, update its trust grade, and confirm its golden test suite is complete. ## Quick Start Run a quality check on the skill at skills/core/knowledge-extraction and report its trust grade and golden test coverage.

Frequently Asked Questions about quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate that a skill actually works as claimed?

Use falsification validation: run the skill against real task inputs (never mock data), include tests expected to fail, and archive the full evidence chain (input, logs, output, comparison, metrics) to test-results/. Trust grades update based on accumulated evidence, not single runs.

What is golden test coverage and how is it measured?

Golden test coverage is the ratio of skills with a complete golden set (GOLDEN_SET.md plus non-empty cases/ and expected/ directories) to total skills. Coverage of 70% or more is acceptable, 30-49% is a P1 improvement target, and below 30% is a P0 systemic quality gap.

How does Bayesian trust scoring work for skills?

The update_trust.py script applies Bayes theorem: each pass or fail result becomes a likelihood that updates the prior trust score into a posterior. Consecutive successes or failures adjust the likelihood, and grades range from High Trust (0.9-1.0) down to Unreliable (below 0.5).

Why must falsification tests avoid mock data?

Mock or synthetic data cannot prove a skill handles real-world variability, so tests built on it are considered invalid evidence. The methodology requires real task inputs and at least one counter-evidence test designed to falsify the skill's claim.

What happens when I pass an empty or invalid skill path?

The request is rejected without executing, returning an error object with context explaining the invalid input and a recovery suggestion pointing to a valid skill directory. No internal state is exposed and no bare exceptions are raised.