firecrawl-crawl

Crawl websites to bulk-extract pages and content with configurable depth and path filters.

Updated Mar 24, 2026

Search Tags:#automation #concurrency #docs #web #data-extraction #crawl #site

Ready to run

npx skills add https://github.com/1thirteeng3/greenmoire --skill firecrawl-crawl-1thirteeng3

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: firecrawl-crawl
Source: https://github.com/1thirteeng3/greenmoire/tree/main/integrations/firecrawl/cli/skills/firecrawl-crawl
Command: npx skills add https://github.com/1thirteeng3/greenmoire --skill firecrawl-crawl-1thirteeng3

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Bulk extract content from a website by crawling pages and collecting data such as URLs and content, saving time when you need multiple pages from a single site or section.

Core Features & Use Cases

Crawl site sections with depth control and path filtering to limit scope.
Execute in parallel to speed up extraction and generate structured output.
Useful for documentation sites or large portals where many pages share a common structure.

Quick Start

Run a site crawl by targeting the base URL and using available options to control depth, limits, and output location.

Frequently Asked Questions about firecrawl-crawl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I bulk extract content from a website section like documentation?▼

Bulk extract content from a website section by crawling pages to collect URLs and data. You can target a base URL and use configurable depth limits and path filtering to scope extraction to specific site sections.

Can I limit the crawl depth and filter paths when extracting web data?▼

You can limit crawl depth and filter paths when extracting web data. The crawl supports configurable depth limits alongside include and exclude path rules to restrict the extraction scope to relevant site areas.

What is the best way to crawl multiple web pages concurrently?▼

The best way to crawl multiple web pages concurrently is to execute parallel extraction during the crawl. This processes many pages simultaneously to speed up data collection and generate structured output.

Does web crawling for data extraction work on large documentation sites?▼

Web crawling for data extraction works effectively on large documentation sites. It collects many pages sharing a common structure, generating structured results suitable for downstream processing.

What format does the site crawl output provide for downstream processing?▼

The site crawl output provides a structured result containing extracted URLs and page content. This format is suitable for immediate downstream processing and data analysis tasks.