audit-skill

Audit Claude Code session transcripts for friction, performance, permission, and accuracy issues.

37|2|Updated Jun 28, 2025
One-click install
npx skills add https://github.com/jellyrock/jellyrock --skill audit-skill-jellyrock
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: audit-skill
Source: https://github.com/jellyrock/jellyrock/tree/main/.claude/skills/audit-skill
Command: npx skills add https://github.com/jellyrock/jellyrock --skill audit-skill-jellyrock

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Skills executed within Claude Code can accumulate hidden friction, bloat token consumption, run on misaligned models, or produce incorrect outputs without any systematic feedback mechanism. This Skill closes that gap by auditing a skill's actual execution trace from its session transcript, surfacing performance anomalies, permission gaps, and accuracy issues that remain invisible during the run itself.

Core Features & Use Cases

  • Four-Dimension Evaluation: Judges runs across friction findings, performance metrics (wall-clock time, token usage, cache-hit ratio, cost estimate), model-fit profile, and output accuracy.
  • Mechanical Extraction + Human Judgment: Runs a deterministic Node.js extractor over JSONL transcripts to produce structured findings, then classifies them with judgment to propose concrete, root-cause-targeted edits.
  • Cross-Session Intelligence: With --last N or --all, aggregates recurring-friction tallies, median/p90 timing, and total cost across sessions to distinguish one-off noise from genuine drift patterns.
  • Anti-Pattern Naming: Every proposed fix names the specific failure shape (bash invocation, tool sequence, or phrasing) so future agents recognize and skip the dead end.

Quick Start

Use the audit-skill skill to evaluate the most recent execution of the pr skill by invoking the slash command /audit-skill pr and review the surfaced findings and edit proposals.

Frequently Asked Questions about audit-skill

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit Claude Code skill transcripts for performance and friction?

You can audit Claude Code skill transcripts by running a deterministic Node.js extractor over JSONL session files to surface friction findings, performance metrics, and output accuracy issues. The audit evaluates wall-clock time, token usage, and cache-hit ratio.

What is skill execution friction and how is it detected from session transcripts?

Skill execution friction refers to hidden inefficiencies like bloat or misaligned models during a run. It is detected by mechanically extracting structured findings from JSONL transcripts and classifying them to propose concrete, root-cause-targeted edits.

How do I track recurring performance anomalies across multiple Claude Code sessions?

You can track recurring performance anomalies across sessions by using cross-session aggregation with flags like --last N or --all. This compiles recurring-friction tallies, median and p90 timing, and total cost to distinguish one-off noise from genuine drift patterns.

Does auditing skill runs require access to specific files or helper scripts?

Auditing skill runs requires the extract-friction.cjs helper script, JSONL transcript access under the ~/.claude/projects/ directory, and the audited skill's SKILL.md frontmatter for model-fit comparison.

How do I validate output accuracy and permission gaps after a skill run?

You validate output accuracy and permission gaps by auditing the skill's actual execution trace from its session transcript. The four-dimension evaluation judges runs across friction, performance metrics, model-fit profile, and output accuracy.

When should I use a post-run skill audit instead of checking results manually?

You should use a post-run skill audit for any non-trivial skill invocation where systematic feedback is needed. It surfaces hidden friction, permission gaps, and accuracy issues that remain invisible during the run itself, naming specific anti-patterns for future avoidance.