What problem does it solve? Extracting content from many pages on the same website one URL at a time is slow and error-prone. This Skill automates bulk extraction by crawling a site or site section, following links up to configurable depth and page limits, and returning structured content for every page. ## Core Features & Use Cases - Bulk Site Crawling: Follow links from a starting URL with configurable --limit, --max-depth, and --max-concurrency controls. - Path Scoping: Restrict crawls with --include-paths and --exclude-paths so only relevant sections like /docs are extracted. - Sync or Async Execution: Use --wait and --progress to block until completion, or receive a job ID for later status polling. - Use Case: You need every page of a product's documentation for offline analysis. Run a crawl scoped to /docs with a page limit, wait for completion, and save the results to .firecrawl/crawl.json. ## Quick Start Ask the assistant to crawl a documentation site section, for example: crawl https://example.com/docs including only the /docs paths with a limit of 50 pages and save the results to a JSON file.
