doc-scraper

Scraping documentation websites converts them into organized reference files by extracting main content, titles, and code blocks using configurable selectors. The skill handles URL patterns, category mappings, and rate limits to produce deterministic, resumable outputs suitable for Claude skills or offline archives.

The scraper supports sites with identifiable main content, titles, and code blocks. It applies to API references, guides, and tutorials with configurable CSS selectors and URL include/exclude patterns to adapt to different site layouts.

Yes, the scraper enforces configurable workflow steps and recoverable checkpoints, enabling deterministic, resumable scrapes. You can pick up from where the process stopped without re-scraping already-collected content.

The skill includes grounding checks that verify target URL accessibility and structure before scraping begins, ensuring the site is reachable and contains the expected content layout.

The skill generates config templates with CSS selectors, URL patterns, category mappings, and rate-limit settings. Output structure, reference organization, and scrape scope are all configurable to match your docs site and archive needs.

Output validation ensures references and metadata are properly structured before saving. The skill produces organized, categorized reference files with validated data ready for use in Claude skills or offline archives.