skills.rest
escotilha
escotilhaCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

firecrawl

Extract and structure public website content into LLM-ready markdown.

Updated Aug 23, 2026
Search Tags:#web scraping#market research#crawling#anti-bot#markdown extraction#proxy rotation#documentation indexing
One-click install
npx skills add https://github.com/escotilha/claude-public --skill firecrawl-escotilha
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl
Source: https://github.com/escotilha/claude-public/tree/main/skills/firecrawl
Command: npx skills add https://github.com/escotilha/claude-public --skill firecrawl-escotilha

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Removes the manual effort of copying, cleaning, and restructuring web pages into a format your AI can reliably read and analyze.

Core Features & Use Cases

  • Web scraping to markdown: Converts web pages into LLM-ready markdown while reducing navigation, headers, footers, and other noise.
  • Crawling and URL discovery: Maps sites and crawls multiple pages with depth and limit controls, supporting documentation indexing and competitive intel.
  • JavaScript rendering and anti-bot robustness: Handles dynamic SPAs and improves extraction success via anti-bot support and proxy rotation.
  • Web search and autonomous research: Combines search with scraping or runs an agent to find and extract relevant data across multiple sources.

Use Cases: Competitive pricing/feature comparisons, market research across multiple articles, indexing documentation/wikis, and building company intelligence (team, funding, revenue) from public web sources.

Quick Start

Run /firecrawl to scrape a target URL and return clean markdown suitable for immediate analysis by an LLM.

Frequently Asked Questions about firecrawl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I turn any website into LLM-ready markdown for analysis?▼

Web scraping converts public website content into clean LLM-ready markdown by stripping navigation, headers, and footers. This Skill structures extracted text for immediate downstream analysis, reducing manual cleaning effort.

What is the best way to crawl full sites and index documentation?▼

Site crawling maps available pages and extracts multiple URLs with depth and limit controls. This documentation indexing approach structures full site content into markdown for competitive intelligence and research.

Does web scraping work with JavaScript rendering and anti-bot proxy rotation?▼

Web scraping handles dynamic SPAs through JavaScript rendering and improves extraction success via anti-bot support and proxy rotation. This ensures robust data retrieval from complex public web sources.

Can I perform web search and autonomous multi-source extraction for market research?▼

Web search combines scraping with autonomous agents to find and extract relevant data across multiple sources. This multi-source extraction builds competitive pricing, feature comparisons, and company intelligence.

Do I need a Firecrawl API key to extract markdown from web pages?▼

Markdown extraction requires a configured FIRECRAWL_API_KEY to utilize Firecrawl MCP tools. These tools enable scrape, crawl, map, search, and agent workflows for structuring public website content.

Why does web scraping fail on dynamic single page applications?▼

Scraping dynamic SPAs fails without JavaScript rendering to execute client-side scripts. Anti-bot handling and proxy rotation bypass protections, ensuring successful markdown extraction from complex pages.

Related Skills

Explore More Data & Analytics Skills→
while-basicwhile-basic

enact/firecrawl

Extract web page content as Markdown or structured data via the Firecrawl API.

Community
Advanced
breethomasbreethomas
16

competitive-research

Analyze competitors by name or URL and generate markdown reports with comparison tables.

Community
Advanced
sixscripts-aisixscripts-ai

deep-research

Automate multi-source web research and synthesize findings with citations.

Community
Advanced
zhayujiezhayujie
46.4k

web-fetch

Fetch web page HTML and extract title and readable text.

Community
Basic