firecrawl-crawl

What problem does it solve?

This Skill automates the process of extracting content from multiple pages of a website, saving significant time and effort compared to manual scraping.

Core Features & Use Cases

• Bulk Content Extraction: Crawl and extract content from an entire website or specific sections.
• Link Following: Automatically follows links to discover and extract content from related pages.
• Filtering: Allows specifying paths to include or exclude, enabling targeted crawls.
• Use Case: Extract all product descriptions from an e-commerce site's /products section, or gather all articles from a blog's /archive directory.

Quick Start

Use the firecrawl crawl skill to extract all pages under the /docs path from https://example.com/docs with a limit of 50 pages, saving the output to .firecrawl/crawl.json.