skills.rest
robwestz
robwestzCOMMUNITY
Explore all skills by this author →
GitHub

kb-scrape

Extract web documentation from URLs into a structured knowledge base.

Updated Apr 9, 2026
Search Tags:#AI assistance#knowledge base#web scraping#API documentation#document capture
One-click install
npx skills add https://github.com/robwestz/memoryporter --skill kb-scrape
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kb-scrape
Source: https://github.com/robwestz/memoryporter/tree/main/skills/skill-kb-scrape
Command: npx skills add https://github.com/robwestz/memoryporter --skill kb-scrape

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires firecrawl-mcp, tavily-mcp, kb-forge-cli, and includes scripts (resource) and metadata (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This skill tackles the challenge of scattered and ephemeral documentation by capturing web content into a permanent, indexed knowledge base for easy retrieval later on.

Core Features & Use Cases

  • Web Documentation Capture: Automatically scrape documentation from URLs into a structured, searchable knowledge base.
  • Use Case: For a project that requires ongoing access to external API documentation, this skill can be used to capture and maintain that information.

Quick Start

Use the "kb-scrape" skill to create a knowledge base of API documentation from the website "https://api.example.com".

Frequently Asked Questions about kb-scrape

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape web documentation into a searchable knowledge base for AI agents?▼

You can scrape web documentation into a searchable knowledge base by extracting content from URLs into a structured format. This skill captures web-sourced API documentation, enabling easy retrieval for AI agents and knowledge workers.

What is the best way to capture external API documentation before it disappears?▼

Capturing external API documentation permanently solves the challenge of ephemeral web content. This skill extracts and indexes web documentation from URLs into a structured knowledge base for ongoing retrieval.

Does this documentation scraping tool work with Firecrawl or Tavily MCP?▼

Yes, this documentation scraping tool works with Firecrawl or Tavily MCP for scraping and indexing. It operates with either dependency to extract web content into a structured knowledge base.

Do I need firecrawl-mcp and tavily-mcp installed to extract web documentation?▼

You need either firecrawl-mcp or tavily-mcp installed to extract web documentation. These dependencies provide the scraping and indexing capabilities required to capture URLs into your structured knowledge base.

Can I use kb-forge-cli to build a knowledge base from web-sourced API documentation?▼

Yes, you can use kb-forge-cli to build a knowledge base from web-sourced API documentation. This skill integrates with kb-forge-cli to structure and maintain captured web content for easy retrieval.

What are the limitations of scraping API documentation into a knowledge base?▼

Scraping API documentation into a knowledge base requires specific MCP dependencies and targets publicly accessible URLs. It is designed for capturing static web documentation rather than interacting with live API endpoints.

Related Skills

Explore More Software Engineering Skills→
smarmen86smarmen86

seo-firecrawl

Crawl and map websites via Firecrawl MCP for SEO audits.

Community
Advanced
robwestzrobwestz

kb-context

Transform knowledge base content into retrievable vector indices for RAG applications.

Community
Intermediate
nathanvalenathanvale
2

firecrawl

Fetch web content and extract structured data via the Firecrawl API.

Community
Advanced
richardnguyen0715richardnguyen0715

scrapling

Scrape static and JavaScript-rendered web content with stealth and Cloudflare bypass.

Community
Advanced