skills.rest
iAziz786
iAziz786COMMUNITY
Explore all skills by this author →
Website𝕏@iAziz786GitHub

firecrawl-map

Maps a website's URLs using a self-hosted Firecrawl instance at localhost:3002.

3|Updated Jul 5, 2021
Search Tags:#automation#cli#firecrawl#web-crawling#urls#site-map
One-click install
npx skills add https://github.com/iAziz786/dotfiles --skill firecrawl-map-iaziz786
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl-map
Source: https://github.com/iAziz786/dotfiles/tree/main/.agents/skills/firecrawl-map
Command: npx skills add https://github.com/iAziz786/dotfiles --skill firecrawl-map-iaziz786

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Discover and list all URLs on a website using a self-hosted Firecrawl instance at localhost:3002, enabling quick site mapping and URL discovery for large domains.

Core Features & Use Cases

  • Discover and list all URLs on a target site.
  • Support for --search to locate specific pages and --limit to cap results.
  • Output formats and guidance to integrate with subsequent crawl or scrape steps.

Quick Start

Map a site by running firecrawl map <url> with optional --search or --limit and save the results to a file.

Frequently Asked Questions about firecrawl-map

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I map all URLs on a website using Firecrawl?▼

To map site URLs, run the firecrawl map command against your target domain. This enumerates all available pages and outputs the site structure, supporting search filters and limits to refine discovery results.

Can I search for specific pages when discovering URLs on a large domain?▼

Yes, you can locate specific pages during site mapping by applying the --search option. This filters the URL discovery process to return only matching pages rather than enumerating the entire domain architecture.

Do I need a self-hosted Firecrawl instance to enumerate site URLs?▼

Yes, enumerating site URLs requires a self-hosted Firecrawl instance running at localhost:3002. You must configure this local environment before executing the map command to discover domain architecture.

How do I limit the number of URLs returned during web crawling discovery?▼

You can cap the number of URLs returned during web crawling discovery by passing the --limit option. This restricts the site mapping output size, preventing excessive results when analyzing large domain architectures.

What is the best way to format mapped URLs for further scraping?▼

The best way to format mapped URLs for further scraping is to use the --json option. This outputs the discovered site structure as structured data, making it easy to integrate with downstream crawl or scrape automation steps.

Related Skills

Explore More Software Engineering Skills→
iAziz786iAziz786
3

firecrawl-scrape

Extract clean Markdown from static and JavaScript-rendered URLs via self-hosted Firecrawl.

Community
Intermediate
abdullah1854abdullah1854
1

crawl-cli

Crawl websites and extract content into Markdown, JSON, or text.

Community
Basic
pypeadaypypeaday
2

drawio

Create and update architecture diagrams via a draw.io MCP server.

Community
Advanced
iAziz786iAziz786
3

firecrawl-crawl

Crawl websites with depth limits and path filters via self-hosted Firecrawl.

Community
Intermediate