What problem does it solve? Extracting content from many pages across a website one URL at a time is slow and tedious. This Skill bulk-extracts content from an entire site or a specific section by automatically following links, so you can gather documentation, articles, or knowledge bases in one operation. ## Core Features & Use Cases - Bulk Site Crawling: Crawl a website following links up to a configurable depth and page limit, saving results to a local JSON file. - Scoped Section Crawls: Restrict crawling to specific paths (e.g., /docs) with --include-paths so you only extract the pages you need. - Sync or Async Execution: Use --wait to block until the crawl finishes, or run asynchronously and poll the job ID for status. - Use Case: You need the entire contents of a product's documentation section for offline analysis. Run a crawl scoped to /docs with a page limit, wait for completion, and get all pages saved under .firecrawl/crawl.json. ## Quick Start Ask the AI to crawl a documentation section, for example: crawl https://example.com limited to the /docs path with a 50-page limit and save the results locally.
