skills.rest
ynulihao
ynulihaoCOMMUNITY
Explore all skills by this author →
GitHub

firecrawl-scraper

Scrape and crawl websites via the Firecrawl API into markdown or structured data.

564|70|Updated Jan 25, 2026
Search Tags:#structured data#web scraping#web crawling#content extraction#javascript rendering#llm ready data
One-click install
npx skills add https://github.com/ynulihao/AgentSkillOS --skill firecrawl-scraper-ynulihao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl-scraper
Source: https://github.com/ynulihao/AgentSkillOS/tree/main/data/skill_seeds/firecrawl-scraper
Command: npx skills add https://github.com/ynulihao/AgentSkillOS --skill firecrawl-scraper-ynulihao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the process of extracting clean, LLM-ready content from websites, overcoming challenges like JavaScript rendering and bot detection.

Core Features & Use Cases

  • Web Scraping: Extract content from single web pages into markdown, HTML, or structured data.
  • Web Crawling: Traverse and scrape entire websites, following links up to a specified depth.
  • Structured Data Extraction: Use AI to pull specific fields from web pages based on a schema.
  • Use Case: Automatically convert an entire documentation website into markdown files to build a knowledge base for a retrieval-augmented generation (RAG) system.

Quick Start

Use the firecrawl-scraper skill to scrape the content from 'https://example.com' and return it as markdown.

Frequently Asked Questions about firecrawl-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from web pages for LLM training?▼

You can extract structured data from web pages for LLMs by using AI to pull specific fields based on a defined schema. This process converts website content into LLM-ready markdown or structured data formats.

What is the best way to scrape JavaScript-rendered websites into markdown?▼

The best way to scrape JavaScript-rendered websites into markdown is by using automated web scraping tools that handle JavaScript rendering and bot bypass. This ensures clean content extraction from dynamic single pages.

Can I crawl an entire website and convert it to markdown for a RAG system?▼

Yes, you can crawl an entire website and convert it to markdown for a RAG system. Full-site crawling traverses websites by following links up to a specified depth to extract all content.

Does web scraping for LLM-ready data handle bot detection bypass?▼

Yes, web scraping for LLM-ready data handles bot detection bypass. The scraping mechanism is designed to overcome challenges like JavaScript rendering and bot detection to ensure successful content extraction.

How do I map URLs from a website before performing full-site crawling?▼

You can map URLs from a website before full-site crawling by using URL mapping features. This identifies all accessible URLs on a target website, allowing you to selectively scrape or crawl specific pages.

What output formats are supported when extracting content from single web pages?▼

Supported output formats for extracting content from single web pages include markdown, HTML, and structured data. You can also capture screenshots to retain the visual layout of the page.

Related Skills

Explore More Software Engineering Skills→
tumftumf
3

firecrawl

Automate web scraping and data extraction via the Firecrawl API.

Community
Advanced
zzafergokzzafergok
1

rag-implementation

Design and build RAG systems grounding LLM outputs in external knowledge sources.

Community
Advanced
SignmanalSignmanal
1

chroma

Store and query vector embeddings with metadata filtering for RAG applications.

Community
Intermediate
appautomatonappautomaton
18

html-to-markdown

Converts HTML or live web pages into structured Markdown with metadata and link inventories.

Official
Intermediate