doc-scraper
CommunityTransform Snowflake docs into AI-ready knowledge instantly.
Authorsfc-gh-dflippo
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates the extraction and organization of Snowflake documentation into structured Markdown with intelligent caching, eliminating manual copying and ensuring content freshness.
Core Features & Use Cases
- Intelligent Caching: Automatically skips recently scraped pages with 7-day expiration to save time and bandwidth.
- Configurable Scraping: Target specific documentation sections with base-path filtering and adjustable spider depth.
- Use Case: Imagine you need the latest Snowflake SQL reference documentation. Use this Skill to scrape the entire SQL reference section, convert it to Markdown with metadata, and generate a comprehensive index for AI access.
Quick Start
Use the doc-scraper skill to extract all migration documentation from docs.snowflake.com into the project's snowflake-docs directory.
Dependency Matrix
Required Modules
requestsbeautifulsoup4lxmlmarkdownifypyyamlpython-frontmatterclicktqdmratelimit
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: doc-scraper Download link: https://github.com/sfc-gh-dflippo/snowflake-dbt-demo/archive/main.zip#doc-scraper Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.