archive-crawler

To surface high-signal content from a personal archive, apply a gold-filter workflow that scans sources like local mounts and Gmail takeouts, triaging valuable items while skipping noise. It automatically maps and categorizes meaningful writing and emails.

The best way to triage Gmail takeout exports is using a schema-generic ingestion workflow that applies a gold-filter to identify emails with substance. It routes high-value items into structured brain pages while discarding irrelevant noise.

Yes, you can scan cloud exports and local mounts without a fixed folder structure because the ingestion workflow is schema-generic. It classifies and routes content dynamically based on content value rather than assuming a predefined directory layout.

An archive crawler handles data provenance by tracking origin details in a manifest during the ingestion workflow. It records session logs and writes categorized brain pages under originals, personal, or ideas directories to maintain full traceability.

Yes, you need an explicit allow-list configured in the scan_paths setting to scan personal archive paths. This safety gate ensures the crawler only accesses explicitly permitted directories, preventing unauthorized ingestion of sensitive data.

A limitation of using a gold-filter for personal data ingestion is that it strictly prioritizes high-signal content like personal writing and ideas, which means it will intentionally skip low-signal noise and may omit routine communications you might still need.