corpus-distiller

Distills large Markdown corpora into thematic deep-reading card series via layered compression.

Updated Aug 15, 2026
One-click install
npx skills add https://github.com/qqlcx5/skills-hub --skill corpus-distiller-qqlcx5
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: corpus-distiller
Source: https://github.com/qqlcx5/skills-hub/tree/main/skills/corpus-distiller
Command: npx skills add https://github.com/qqlcx5/skills-hub --skill corpus-distiller-qqlcx5

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Facing hundreds to tens of thousands of Markdown articles or notes, users cannot read them one by one and have no map of what is worth deep reading. This Skill compresses the entire corpus through a five-layer funnel and outputs a structured series of thematic deep-reading cards instead of a single summary. ## Core Features & Use Cases - Layered Compression Funnel: Surveys the corpus with a shell script, clusters filenames by domain, deep-reads only benchmark files, then cross-maps concepts across domains to surface axioms versus noise. - Thematic Card Series Output: Produces a master topic table (00 主题总表.md) plus individual topic cards with fixed structure: cold-start case, core axiom, three pillar books, counterexample, cross-disciplinary mapping, and a curated reading list of real in-corpus files. - Parallel Sub-Agent Dispatch: Ships reference templates for dispatching Explore sub-agents across batches of roughly 500 filenames each, with a ten-domain taxonomy and compact reporting format. - Use Case: A user with 5,000 Markdown book notes asks to distill the library; the Skill batches the filenames, dispatches parallel drill-down agents, synthesizes 12-20 cross-domain themes, and generates one deep-reading card per theme with verifiable in-corpus sources. ## Quick Start Ask the agent to distill your Markdown notes directory into a thematic deep-reading card series using the corpus-distiller workflow.

Frequently Asked Questions about corpus-distiller

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I summarize thousands of Markdown notes without reading each one?

Use a layered compression approach: cluster filenames first, read only skeletons (first lines and headings), then deep-read a few benchmark files per domain. This Skill automates that funnel and outputs thematic cards instead of per-file summaries.

How to batch process a large Markdown corpus with sub-agents?

Run the survey script to split filenames into batches of about 500, then dispatch one Explore sub-agent per batch using the dispatch-prompts.md templates. Each sub-agent returns a compact domain summary under 1500 words for the main agent to cross-map.

What is the minimum corpus size for batch distillation?

Corpora under 200 Markdown files skip batching entirely; the main agent clusters the full filename list directly. The survey script detects this automatically and only splits larger corpora into parallel batches.

Does corpus distillation work with non-English or single-domain notes?

Yes, the ten-domain taxonomy is a default reference only. If the corpus is single-domain, such as all technical documents, replace the taxonomy in the dispatch prompt with a self-consistent custom classification.

Why does the distillation forbid information outside the corpus?

All book titles, cases, and data in the generated cards must come from real files inside the library to keep the reading list actionable. Fabricated references would break the core promise that every card maps to content the user actually owns.