archive-crawler

Scans allow-listed personal archives and ingests high-value writing into brain pages.

Updated Aug 17, 2026
One-click install
npx skills add https://github.com/AMC-JTC/gbrain-1 --skill archive-crawler-amc-jtc
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: archive-crawler
Source: https://github.com/AMC-JTC/gbrain-1/tree/main/plugin/skills/archive-crawler
Command: npx skills add https://github.com/AMC-JTC/gbrain-1 --skill archive-crawler-amc-jtc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Personal archives scattered across Dropbox, Backblaze B2, Gmail takeouts, and old hard drives contain valuable writing, ideas, and correspondence that are never revisited. This Skill systematically explores those archives, filters out noise, and surfaces the content worth preserving in a structured knowledge base. ## Core Features & Use Cases - Safety-gated scanning: Refuses to run unless an explicit archive-crawler.scan_paths: allow-list is set in gbrain.yml, preventing accidental ingestion of sensitive files like tax documents or medical records. - Gold filtering and triage: Applies a keep/skip filter to separate personal writing, ideas, and relationship material from system files, binaries, and bulk mail, tracking every item's status in a manifest page. - Multi-format ingestion: Handles plain text, HTML, Markdown, .mbox email archives, .doc/.docx, .pst Outlook files, and compressed archives, filing ingested content into originals/, personal/, or ideas/ per the user's filing rules. - Use Case: Point the Skill at an old Dropbox archive of letters and journals; it maps the tree, proposes a priority queue, shows high-value items one at a time, captures your exact reactions, and creates cross-linked brain pages for each keeper. ## Quick Start Add the folders you want scanned to the archive-crawler.scan_paths: list in gbrain.yml, then ask the agent to crawl your archive and surface the writing worth keeping.

Frequently Asked Questions about archive-crawler

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scan my Dropbox or old hard drive for valuable writing?

Add the target folders to the archive-crawler.scan_paths allow-list in gbrain.yml, then ask the agent to crawl your archive. It maps the tree, ranks folders by likely value, and presents high-signal items one at a time for review and ingestion.

How do I extract text from mbox, PST, and doc email archives?

The Skill uses Python's mailbox module for .mbox files, readpst for valid Outlook .pst files (after checking the !BDN magic bytes), and antiword or catdoc for legacy .doc files. Modern .docx files are parsed by unzipping and reading word/document.xml.

Why does archive-crawler refuse to run on my files?

The Skill hard-refuses to run when archive-crawler.scan_paths is missing or empty in gbrain.yml. This is a deliberate safety fence so the agent never infers which directories are safe to scan; you must explicitly allow-list paths first.

Can I exclude sensitive folders like finances or medical records?

Yes. In addition to the mandatory scan_paths allow-list, you can set an optional deny_paths list inside it, such as ~/Documents/finances/ or ~/Documents/medical/, and those locations will be skipped during the crawl.

What file types does the archive crawler skip automatically?

The gold filter skips system files, configs, node_modules, binary blobs like images and video, receipts, tax documents, spam, newsletters, and corrupted or null files. Images are noted with metadata but not shown unless requested.