skills.rest
dennislee928
dennislee928COMMUNITY
Explore all skills by this author →
WebsiteGitHub

firecrawl-scraper

Scrapes and crawls websites into LLM-ready markdown or JSON.

54|Updated Nov 19, 2025
Search Tags:#automation#data extraction#web scraping#web crawling#markdown conversion#llm data
One-click install
npx skills add https://github.com/dennislee928/Ethic-Latex --skill firecrawl-scraper-dennislee928
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl-scraper
Source: https://github.com/dennislee928/Ethic-Latex/tree/main/.claude/skills/firecrawl-scraper
Command: npx skills add https://github.com/dennislee928/Ethic-Latex --skill firecrawl-scraper-dennislee928

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the process of extracting and structuring data from websites, making it readily usable for AI models and applications, thereby saving significant development time and effort.

Core Features & Use Cases

  • Web Scraping & Crawling: Scrape single pages or crawl entire websites.
  • Data Transformation: Convert raw web content into LLM-ready markdown or structured JSON.
  • Use Case: Automatically gather product information from an e-commerce site, convert documentation websites into a knowledge base for RAG, or extract contact details from a list of company websites.

Quick Start

Use the firecrawl-scraper skill to scrape the main content from 'https://example.com' into markdown format.

Frequently Asked Questions about firecrawl-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert web page content into markdown for LLM data processing?▼

Web scraping converts raw website content into LLM-ready markdown or structured JSON formats. This skill automates single-page scraping and full-site crawling to extract content directly usable by AI models.

Can I crawl an entire documentation site to build a knowledge base for RAG?▼

Full-site crawling extracts content from entire documentation websites to build a knowledge base for RAG. This skill transforms raw web pages into structured markdown or JSON suitable for retrieval augmented generation.

How does web scraping handle JavaScript rendering and anti-bot protections?▼

Web scraping handles JavaScript rendering and bypasses anti-bot protections automatically. This enables programmatic data extraction from dynamic websites that require client-side script execution to load content.

What is the best way to extract structured data from e-commerce product pages?▼

Structured data extraction pulls specific information like product details from e-commerce sites into JSON format. An autonomous agent-based approach gathers and formats this web data without manual intervention.

Does this web scraping tool support parsing documents alongside standard HTML pages?▼

Document parsing is supported alongside standard HTML page scraping for diverse web data needs. This allows the extraction of content from various file types and formats it into LLM-ready markdown or JSON.

Why use autonomous agents for web crawling instead of static data extraction?▼

Autonomous agent-based web crawling navigates and gathers data dynamically across multiple pages. Unlike static extraction, agents handle complex site structures and JavaScript rendering to retrieve LLM-ready content automatically.

Related Skills

Explore More Software Engineering Skills→
JuanMS20JuanMS20

github-auth

Configure GitHub authentication for Hermes Agent using git, gh CLI, or SSH credentials.

Community
Intermediate
bugrabilgebugrabilge
10

rag-implementation

Integrate vector databases and semantic search to build RAG pipelines for LLM applications.

Community
Advanced
DrLuggelsDrLuggels
3

llm-evaluation

Evaluate LLM applications with automated metrics, human feedback, and benchmarking.

Community
Advanced
appautomatonappautomaton
18

html-to-markdown

Converts HTML or live web pages into structured Markdown with metadata and link inventories.

Official
Intermediate