firecrawl-scraper

Scrape and crawl websites via the Firecrawl API into markdown or structured data.

564|70|Updated Jan 25, 2026

Search Tags:#structured data #web scraping #web crawling #content extraction #javascript rendering #llm ready data

Ready to run

npx skills add https://github.com/ynulihao/AgentSkillOS --skill firecrawl-scraper-ynulihao

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: firecrawl-scraper
Source: https://github.com/ynulihao/AgentSkillOS/tree/main/data/skill_seeds/firecrawl-scraper
Command: npx skills add https://github.com/ynulihao/AgentSkillOS --skill firecrawl-scraper-ynulihao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the process of extracting clean, LLM-ready content from websites, overcoming challenges like JavaScript rendering and bot detection.

Core Features & Use Cases

Web Scraping: Extract content from single web pages into markdown, HTML, or structured data.
Web Crawling: Traverse and scrape entire websites, following links up to a specified depth.
Structured Data Extraction: Use AI to pull specific fields from web pages based on a schema.
Use Case: Automatically convert an entire documentation website into markdown files to build a knowledge base for a retrieval-augmented generation (RAG) system.

Quick Start

Use the firecrawl-scraper skill to scrape the content from 'https://example.com' and return it as markdown.

Frequently Asked Questions about firecrawl-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract structured data from web pages for LLM training?▼

You can extract structured data from web pages for LLMs by using AI to pull specific fields based on a defined schema. This process converts website content into LLM-ready markdown or structured data formats.

What is the best way to scrape JavaScript-rendered websites into markdown?▼

The best way to scrape JavaScript-rendered websites into markdown is by using automated web scraping tools that handle JavaScript rendering and bot bypass. This ensures clean content extraction from dynamic single pages.

Can I crawl an entire website and convert it to markdown for a RAG system?▼

Yes, you can crawl an entire website and convert it to markdown for a RAG system. Full-site crawling traverses websites by following links up to a specified depth to extract all content.

Does web scraping for LLM-ready data handle bot detection bypass?▼

Yes, web scraping for LLM-ready data handles bot detection bypass. The scraping mechanism is designed to overcome challenges like JavaScript rendering and bot detection to ensure successful content extraction.

How do I map URLs from a website before performing full-site crawling?▼

You can map URLs from a website before full-site crawling by using URL mapping features. This identifies all accessible URLs on a target website, allowing you to selectively scrape or crawl specific pages.

What output formats are supported when extracting content from single web pages?▼

Supported output formats for extracting content from single web pages include markdown, HTML, and structured data. You can also capture screenshots to retain the visual layout of the page.