consolidation

Group overlapping skill documents into ranked merge, migrate, and cross-reference reports.

Updated Apr 20, 2026

Search Tags:#knowledge base #deduplication #semantic similarity #repo maintenance #documentation governance #graph clustering #embedding cache

Ready to run

npx skills add https://github.com/thistleknot/skills --skill consolidation-thistleknot

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: consolidation
Source: https://github.com/thistleknot/skills/tree/main/consolidation
Command: npx skills add https://github.com/thistleknot/skills --skill consolidation-thistleknot

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires fastapi, uvicorn, networkx, numpy, scikit-learn, python-louvain, matplotlib, requests.

What problem does it solve?

Consolidation eliminates drift in a growing skill library by identifying duplicated or overlapping content across multiple skill documents and proposing the correct governance action.

Core Features & Use Cases

Triplet-based similarity: Computes pairwise overlap from extracted subject–predicate–object triplets (with optional NLI soft matching behavior).
Greedy nearest-neighbor chain decomposition: Groups skills into ordered chains using a semantic floor (τ) to isolate meaningful merge candidates from distinct topics.
Prescription layer: Classifies adjacent links in each chain as MERGE (deduplicate), migrate, or cross-reference, then generates ranked group reports suitable for a living knowledge base.
Embedding queue decoupling: Uses an async embedding queue server so consolidation runs don’t block on embedding recomputation and can leverage cached checkpoint data.

Quick Start

Run consolidation to produce consolidation_report.md and ranked merge/migrate/xref prescriptions for the current skills corpus.

Frequently Asked Questions about consolidation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I deduplicate overlapping documentation in a knowledge base?▼

To deduplicate overlapping documentation in a knowledge base, the process extracts subject–predicate–object triplets, computes pairwise semantic similarity, and emits ranked group reports with merge, migrate, or cross-reference prescriptions.

What is semantic similarity clustering used for in repo maintenance?▼

Semantic similarity clustering in repo maintenance identifies duplicated or overlapping content across skill documents by decomposing nearest-neighbor chains above a similarity threshold to isolate meaningful merge candidates.

When should I run documentation governance on a skill library?▼

You should run documentation governance during skill-library growth, after batches of edits, or when search results appear contradictory across multiple files, ensuring drift is eliminated by proposing correct governance actions.

Does consolidation work with networkx and scikit-learn for graph clustering?▼

Yes, consolidation uses networkx for graph clustering and scikit-learn for semantic similarity computations, leveraging these dependencies to build pairwise similarity matrices and decompose them into nearest-neighbor chains.

What's the best way to generate merge-ready reports for semantically overlapping documents?▼

The best way to generate merge-ready reports for overlapping documents is extracting triplets with optional NLI soft matching, classifying adjacent links by similarity thresholds, and emitting ranked group documentation artifacts.

Why do consolidation runs not block on embedding recomputation?▼

Consolidation runs do not block on embedding recomputation because the system uses an async embedding queue server that decouples the process, allowing it to leverage cached checkpoint data instead of recalculating synchronously.