archify-tuning

Evaluates and improves Archify diagram generation through controlled experiments, trace analysis, and replay benchmarks.

83.7k|5.7k|Updated Apr 15, 2026
Ready to run
npx skills add https://github.com/tt-a1i/archify --skill archify-tuning-tt-a1i
💬 Prompt for ChatGPT / Claude Chat▼
Please help me install this Agent Skill.
Skill: archify-tuning
Source: https://github.com/tt-a1i/archify/tree/main/.agents/skills/archify-tuning
Command: npx skills add https://github.com/tt-a1i/archify --skill archify-tuning-tt-a1i

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Improving an AI diagram-authoring pipeline requires evidence, not guesswork. This Skill provides a structured experimentation loop for measuring and improving Archify's diagram accuracy, visual quality, and authoring speed, using real agent traces, fixed-input replays, and screenshot comparisons instead of anecdotal judgments. ## Core Features & Use Cases - Bounded experiment loop: Record a hypothesis, evidence criteria, and budget in a journal, then attribute failures to specific causes (placement, routing, diagnostics, instructions, or unsuitable rules) before changing the smallest responsible scope. - Benchmark replay and comparison: Render frozen first-draft benchmarks against base and head Archify revisions with replay.mjs, then capture side-by-side screenshots with shots.mjs to detect regressions and visual changes. - Trace collection and analysis: Gather per-diagram-type statistics (tool calls, finalize failures, diagnostic codes, lost content) from Devin CLI history or a normalized trace export using collect.mjs. - Use Case: After changing the workflow router, run a replay against the benchmark corpus to confirm no previously passing draft regressed, then screenshot the changed outputs to verify the whole diagram still reads well. ## Quick Start Ask the agent to resume the archify-tuning journal and run a bounded experiment comparing the current Archify revision against the recorded baseline using the replay and screenshot tools.

Frequently Asked Questions about archify-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test whether an Archify change regressed diagram rendering?▼

Run replay.mjs with --base pointing at the baseline Archify tree; it renders every benchmark first draft with both revisions and reports pass/fail flips and changed SVG output per type. It exits 1 when a draft that passed on base fails on head.

How do I collect agent traces for diagram generation runs?▼

Run collect.mjs with the round directory; it reads Devin CLI session history or a normalized trace export via --trace-export, then reports tool calls, finalize outcomes, diagnostic codes, and content lost between first draft and final candidate.

Can I use this tuning workflow with agents other than Devin CLI?▼

Yes. Export the sessions with the host's supported tools and normalize them to the schemaVersion 1 trace format described in references/trace-export.md, then pass the file to collect.mjs with --trace-export. The collector never reads other agents' private databases.

What tools are required to run the replay screenshot comparison?▼

shots.mjs requires Chrome or Chromium, discoverable automatically or via the ARCHIFY_CHROME environment variable, plus Node 22.13 or later. It captures base and head renders at 1440px width and writes a side-by-side index.html.

Why does a passing finalize check not prove a good diagram?▼

Machine acceptance only covers the rules it encodes; accuracy against source facts and whole-diagram readability require separate audit and visual inspection. The skill treats tests as candidates for improvement and keeps machine pass separate from visual acceptance.

Where does the tuning data live and why outside the repository?▼

Data lives in $ARCHIFY_TUNING_HOME, defaulting to ~/.local/share/archify-tuning, holding journal.md, bench/, rounds/, and replays/. It stays outside the repository because drafts carry content from source projects that may be private and must never be committed.