firecrawl-map

Discovers and lists all URLs on a website with optional search filtering.

2|Updated Mar 29, 2015
One-click install
npx skills add https://github.com/ovisan/dotfiles --skill firecrawl-map-ovisan
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: firecrawl-map
Source: https://github.com/ovisan/dotfiles/tree/main/.grok/skills/firecrawl-map
Command: npx skills add https://github.com/ovisan/dotfiles --skill firecrawl-map-ovisan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When you know which website contains the information you need but not the exact page, manually browsing large sites is slow and unreliable. This Skill maps an entire domain to enumerate its URLs, optionally filtering them by a search query. ## Core Features & Use Cases - Site URL Discovery: Enumerate all URLs on a domain with configurable limits, sitemap handling, and subdomain inclusion. - Search Filtering: Use the --search option to locate a specific page (e.g., an authentication docs page) within a large site. - Structured Output: Export results as plain text or JSON to a file for downstream scraping or crawling. - Use Case: Run firecrawl map https://docs.example.com --search "auth" to find the exact authentication documentation URL, then scrape that page directly. ## Quick Start Ask the AI to map a website and find the URL for a specific topic, for example: map the docs site and find the page about authentication.

Frequently Asked Questions about firecrawl-map

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a specific page on a large website?

Use the firecrawl map command with the --search option to filter discovered URLs by a query. For example, run firecrawl map with the site URL and --search "authentication" to locate the relevant page, then scrape that URL.

How to list all URLs on a website with Firecrawl?

Run firecrawl map followed by the site URL, optionally adding --limit to cap results and --json with -o to save structured output to a file. The command enumerates URLs using the site's sitemap and link discovery.

What is the difference between Firecrawl map, scrape, and crawl?

Map discovers and lists URLs without extracting content, scrape extracts content from a single known URL, and crawl bulk-extracts content across many pages. Map is typically used first to find URLs, followed by scrape or crawl.

Can Firecrawl map include subdomains in results?

Yes, pass the --include-subdomains flag to the map command to include URLs from subdomains of the target site. Without this flag, only URLs on the primary domain are returned.

When should I use crawl instead of map and scrape?

Use crawl when you need to bulk-extract content from many pages rather than just discovering URLs. Map plus scrape is better when you only need one or a few specific pages found via search filtering.