skills.rest
webdevarif
webdevarifCOMMUNITY
Explore all skills by this author →
Website𝕏@webdevarifGitHub

firecrawl-map

Discover and list all URLs on a website with optional search filtering.

Updated Apr 18, 2026
Search Tags:#cli#firecrawl#web scraping#crawling#site map#url discovery#search filter
One-click install
npx skills add https://github.com/webdevarif/claude-skills --skill firecrawl-map-webdevarif
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl-map
Source: https://github.com/webdevarif/claude-skills/tree/main/firecrawl-map
Command: npx skills add https://github.com/webdevarif/claude-skills --skill firecrawl-map-webdevarif

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When you know which website contains the information you need but not the exact page, manually browsing large sites is slow and unreliable. This Skill maps an entire domain and lists its URLs so you can locate specific pages quickly. ## Core Features & Use Cases - Full URL Discovery: Enumerate all URLs on a site with configurable limits, sitemap handling, and subdomain inclusion. - Search Filtering: Use the --search option to find a specific page (e.g., "authentication") within a large site without listing everything. - Structured Output: Export results as plain text or JSON to a file for downstream scraping or crawling. - Use Case: You need the API authentication docs from a large documentation site. Run map with --search "auth" to find /docs/api/authentication, then scrape that exact URL. ## Quick Start Ask the AI to map the target website and filter for the topic you need, for example: "Map https://docs.example.com and find the page about authentication."

Frequently Asked Questions about firecrawl-map

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a specific page on a large website?▼

Use the firecrawl map command with the --search option to filter URLs by a query term. For example, mapping a docs site with --search "authentication" returns only matching pages like /docs/api/authentication.

How to list all URLs on a website with Firecrawl?▼

Run firecrawl map with the target URL and set --limit to control the maximum number of URLs returned. Add --json and -o to export the full list as a JSON file for further processing.

What is the difference between Firecrawl map, scrape, and crawl?▼

Map discovers URLs on a site without extracting content, scrape extracts content from a single known URL, and crawl bulk-extracts content across many pages. A common pattern is map to find the URL, then scrape it.

Can Firecrawl map include subdomains and sitemaps?▼

Yes, the --include-subdomains flag adds subdomain URLs to results, and --sitemap accepts include, skip, or only to control how sitemaps are used during discovery.

When should I use crawl instead of map?▼

Use crawl when you need the actual page content from many URLs rather than just a list of links. Map only discovers URLs; if your goal is bulk content extraction, crawl or map-plus-scrape is the better fit.

Related Skills

Explore More Software Engineering Skills→
ZJU-REALZJU-REAL
195

firecrawl-map

Discover and filter website URLs using the Firecrawl command-line interface.

Official
Basic
redbananastudiosredbananastudios

firecrawl

Scrapes, crawls web content via Firecrawl API with schema-based JSON extraction.

Community
Intermediate
The-Focus-AIThe-Focus-AI

firecrawl-download

Downloads entire websites as local markdown, screenshot, and multi-format files.

Official
Basic
MattSenterMattSenter

senternet-site-robots

Creates robots.txt files that allow crawlers and reference the sitemap.

Community
Basic