Partition large Markdown and JSON documents into configurable shard groups.
npx skills add https://github.com/eaglhuang/cocos_new --skill doc-shard-manager
Please help me install this Agent Skill. Skill: doc-shard-manager Source: https://github.com/eaglhuang/cocos_new/tree/main/.github/skills/doc-shard-manager Command: npx skills add https://github.com/eaglhuang/cocos_new --skill doc-shard-manager
Large documents (Markdown/JSON) often exceed token limits, forcing costly workarounds. This skill introduces shard groups to split big files into small, independently loadable parts and maintain a lightweight index stub for efficient reassembly and validation.
Run a shard operation on a new large file to split it into shard folders and update the index.
High-intent search queries and answers about installing and using this skill.
To split large Markdown and JSON files and avoid exceeding token limits, you can partition them into shard groups. This creates small, independently loadable parts and maintains a lightweight index stub for efficient reassembly and progressive loading.
Document sharding is the process of breaking large documents into smaller, independently loadable parts to facilitate efficient indexing. It helps manage large files by maintaining a lightweight index stub, enabling incremental loading and validation without processing the entire file at once.
You configure shard groups for large documents using a .shardrc.json file. This configuration file orchestrates shifting large content into shard directories, setting thresholds for auto-splitting, and defining parameters for rebuilds and validations.
Yes, you can automatically split files when shards exceed a specific size threshold by using the auto-split command. The system performs safety checks and scans for new large files to automate the partitioning process without manual intervention.
Available commands for managing and validating document shards include shard, rebuild-index, validate, status, and auto-split. These commands support initial splitting, index rebuilding, validation, and status checks across Markdown and JSON sources.
Yes, this large-file management approach works with both Markdown and JSON sources. The sharding architecture supports workflows involving initial splitting, index rebuilding, validation, and status checks specifically tailored for these file formats.
ldk-hubSplit large markdown documents into shard files by level-2 sections.
iamsteveholmesCapture conversation story ideas into backlog stub files with index entries.
clovernguyen1010-ship-itSplit large Markdown documents into section-based shard files with an index.md.
sfinesSplit large markdown documents into smaller files by level 2 sections.
