What problem does it solve? Editing the data layer or crawl orchestration of this Node.js newsletter crawler without knowing the schema, queue semantics, or dedup rules leads to subtle bugs like lost backlog, zombie jobs, or broken delta anchors. This Skill injects the verified internal knowledge needed to safely modify src/db.js, src/crawl.js, src/curate.js, src/events.js, and src/index.js. ## Core Features & Use Cases - Schema and statements reference: Covers all tables (sources, pages, articles, frontier, events, searches), the single stmts object with claimNext/enqueue/insertArticle, and idempotent ensureColumn migrations. - Frontier and orchestration rules: Documents the pending/in_progress/done/failed state machine, resume via resetInProgress, the work-phase job clock with real abort, batched event writes, and the concurrency loop with streaming verify. - Dedup, dates, and pagination: Explains canonical URL plus content_hash dedup, the --since date floor, per-source capture cursors, and deterministic known-url pagination stops. - Use Case: When fixing a bug where re-crawling re-downloads already-known articles, consult this Skill to find the isUrlKnown four-branch guard and the maxArticleRunId delta anchor before touching crawl.js. ## Quick Start Ask the AI to explain how the frontier queue resumes after a crash and which statements in src/db.js to modify for a new article column.
