skills.rest
webdevarif
webdevarifCOMMUNITY
Explore all skills by this author →
Website𝕏@webdevarifGitHub

firecrawl-knowledge-ingest

Extract articles and metadata from documentation portals using Firecrawl browser navigation.

Updated Apr 18, 2026
Search Tags:#documentation#browser automation#firecrawl#knowledge base#web scraping#markdown extraction
One-click install
npx skills add https://github.com/webdevarif/claude-skills --skill firecrawl-knowledge-ingest-webdevarif
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: firecrawl-knowledge-ingest
Source: https://github.com/webdevarif/claude-skills/tree/main/firecrawl-knowledge-ingest
Command: npx skills add https://github.com/webdevarif/claude-skills --skill firecrawl-knowledge-ingest-webdevarif

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Scraping modern documentation portals is difficult when content is behind JavaScript rendering, login walls, pagination, or dynamic navigation. This Skill uses the Firecrawl browser to navigate these portals and extract structured knowledge base content. ## Core Features & Use Cases - Browser-Based Navigation: Opens portals, follows sidebar links, pagination, load-more controls, and search to discover article URLs. - Structured Extraction: Scrapes article content as markdown and captures metadata such as title, section, last updated date, author, and tags. - Auth-Gated Support: Handles login-required portals and JS-heavy docs that simple HTTP scrapers cannot reach. - Use Case: Ingest a paginated support knowledge base into a single JSON file with sections, article content, and source URLs for use in a RAG pipeline. ## Quick Start Use the firecrawl-knowledge-ingest skill to extract all articles from the docs portal at the given URL into structured markdown.

Frequently Asked Questions about firecrawl-knowledge-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a JavaScript-heavy documentation site?▼

Use Firecrawl browser navigation to render JS-heavy documentation sites, follow sidebar links and pagination, and scrape each article as markdown. This handles content that plain HTTP scrapers cannot render.

How to extract articles from a login-gated knowledge base?▼

Firecrawl browser navigation supports authenticated portals, so it can open login-gated knowledge bases and extract article content. Provide credentials through the browser session and respect authentication boundaries during extraction.

Does Firecrawl support paginated help centers?▼

Yes, the workflow follows pagination, next links, load-more controls, and search to discover all article URLs in a help center. It tracks extraction progress and records any failed or restricted pages.

What output format does Firecrawl knowledge ingestion produce?▼

Output is JSON or markdown containing source, url, extractedAt, totalArticles, and sections with article title, url, section, content, and metadata. Code examples, tables, and formatting are preserved while navigation chrome is stripped.

What are the limitations of browser-based docs scraping?▼

Browser-based scraping requires a valid Firecrawl API key and may hit access restrictions on some pages, which are logged as failed or restricted. Very large portals may need a max_pages limit to control extraction scope.

Related Skills

Explore More Software Engineering Skills→
ironkid90ironkid90

firecrawl-knowledge-ingest

Extract structured data from web knowledge bases using Firecrawl.

Community
Advanced
sylin-orgsylin-org
4

koan-ai-integration

Integrate chat endpoints, embeddings, and RAG workflows with vector search.

Official
Advanced
harperaaharperaa
30

ingest

Ingest URLs or files and extract deduplicated insights with source attribution.

Community
Intermediate
avalonresetavalonreset

seo-firecrawl

Crawl, map, and scrape websites for SEO audit analysis.

Community
Intermediate