archive-crawler

To safely crawl personal archives, you must define an explicit allow-list of scan paths in your configuration file. This safety gate prevents accidental ingestion of sensitive data by strictly limiting the crawl to approved directories.

Extracting writing and ideas from takeout or local drives requires a schema-driven crawler that catalogs files and generates a structured manifest. This manifest surfaces high-signal content for your review and triage before ingestion.

To route discovered content into originals, personal, or ideas folders, apply explicit filing rules during the crawl. The pipeline validates content and routes it accordingly, producing a deterministic audit trail for each file.

The crawler works with cloud backups and local hard drives but refuses to run unless an explicit allow-list is configured. This ensures only approved file trees are scanned, preventing overreach into unapproved directories.

Your archive crawl refuses to start because a strict safety gate requires an explicit scan paths allow-list in your configuration. Without this setting, the crawler prevents execution to avoid accidental ingestion of sensitive data.

Before scanning Dropbox archives, you need to configure an explicit allow-list of scan paths in your configuration file. This prerequisite establishes a safe boundary for the crawler to discover writing, ideas, and relationships.