skills.rest
yoshisada
yoshisadaCOMMUNITY
Explore all skills by this author →
GitHub

kiln-research

Automate baseline-vs-candidate plugin research and emit a markdown report.

Updated Mar 26, 2026
Search Tags:#automation#research#benchmarking#plugin#report#reproducibility#kiln
One-click install
npx skills add https://github.com/yoshisada/ai-repo-template --skill kiln-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kiln-research
Source: https://github.com/yoshisada/ai-repo-template/tree/main/plugin-kiln/skills/kiln-research
Command: npx skills add https://github.com/yoshisada/ai-repo-template --skill kiln-research

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Drive the existing kiln-test substrate to compare a baseline plugin arm against a candidate across a declared corpus, generating a comparative markdown report at .kiln/logs/research-<uuid>.md.

Core Features & Use Cases

  • Automates running the baseline and candidate arms against a corpus to produce a side-by-side evaluation.
  • Emits a deterministic, centralized report and keeps per-arm scratch data for post-mortem analysis.
  • Supports optional per-axis direction gating via PRD frontmatter to enforce quality constraints when requested.

Quick Start

Invoke kiln-research with a baseline plugin directory, a candidate plugin directory, and a corpus directory to generate the comparative report.

Frequently Asked Questions about kiln-research

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate baseline versus candidate benchmarking for plugin research?▼

Automating baseline versus candidate benchmarking involves running both plugin arms against a declared corpus and generating a comparative markdown report. kiln-research drives the existing kiln-test substrate to produce this side-by-side evaluation automatically, emitting deterministic results.

What is comparative baseline-vs-candidate research and when do I need it?▼

Comparative baseline-vs-candidate research evaluates a new candidate plugin against an existing baseline across a corpus. You need this research workflow when validating plugin improvements, ensuring reproducibility, and enforcing per-axis quality constraints during benchmarking.

How do I enforce per-axis quality direction constraints during benchmarking?▼

To enforce per-axis quality direction constraints during benchmarking, use the optional --prd frontmatter gating. This feature enforces specific quality constraints declared in your PRD when comparing the baseline and candidate plugin arms.

Where are benchmarking research reports and scratch data stored on failure?▼

Benchmarking research reports are written to .kiln/logs/research-<uuid>.md, while per-arm scratch directories for post-mortem analysis persist under /tmp/kiln-test-<uuid>/ on failure, ensuring deterministic execution and centralized logging.

Does kiln-research require any external dependencies to run plugin benchmarks?▼

kiln-research requires no external dependencies to run plugin benchmarks. It relies solely on wheel's harness for deterministic execution by resolving the wheel install directory via a dual-layout traversal within the existing kiln-test substrate.

What's the best way to compare a baseline plugin against a candidate across a corpus?▼

The best way to compare a baseline plugin against a candidate is invoking kiln-research with baseline, candidate, and corpus directories, which generates a centralized comparative markdown report while keeping per-arm scratch data for post-mortem analysis.

Related Skills

Explore More Education & Research Skills→
yoshisadayoshisada

wheel-init

Scaffold the wheel runtime with workflows and hook configuration.

Community
Intermediate
yoshisadayoshisada

kiln-test

Run end-to-end plugin-skill tests in real Claude subprocesses with TAP output.

Community
Intermediate
yoshisadayoshisada

kiln-distill

Bundle open issues, feedback, and roadmap items into feature PRDs.

Community
Advanced
joewinkejoewinke
248

jat-verify

Verify UI features in a real browser using browser automation tools.

Community
Intermediate