skills.rest
VitorAndTxr
VitorAndTxrCOMMUNITY
Explore all skills by this author →
GitHub

api-doc-scraper

Crawl API documentation sites and generate corpus.md and nav-map.json.

Updated Feb 7, 2026
Search Tags:#documentation#authentication#api#reference#scraper#endpoints#skill-creator
One-click install
npx skills add https://github.com/VitorAndTxr/omni-sw --skill api-doc-scraper
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: api-doc-scraper
Source: https://github.com/VitorAndTxr/omni-sw/tree/main/skills/api-doc-scraper
Command: npx skills add https://github.com/VitorAndTxr/omni-sw --skill api-doc-scraper

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests, beautifulsoup4, and includes scripts (resource) and references (resource) components.

What problem does it solve?

api teams struggle to translate scattered API documentation into a unified, interactive reference that can be queried during development and debugging.

Core Features & Use Cases

  • Web-based API documentation scraping to build a structured, searchable corpus
  • Generate a Claude Code skill that serves as an interactive API reference for developers
  • Use cases include onboarding, API exploration, and rapid debugging with up-to-date docs

Quick Start

Use the API docs scraper to convert an API docs site into a new interactive API-reference skill.

Frequently Asked Questions about api-doc-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert web API documentation into a searchable corpus?▼

You can convert web API documentation into a searchable corpus by providing a root URL to a Python scraper, which uses requests and BeautifulSoup to generate a staged corpus.md and nav-map.json for queryable endpoint references.

Can I scrape API docs to build an interactive reference for endpoints and authentication?▼

Yes, scraping API docs captures endpoints, authentication, models, and errors by crawling a root URL, then outputs structured references consumed by a skill generator pipeline to enable rapid interactive Q&A.

Do I need Python and BeautifulSoup to scrape API documentation into a skill corpus?▼

Yes, scraping API documentation into a skill corpus requires Python 3.x along with the requests and beautifulsoup4 dependencies to fetch and parse web pages into structured corpus files.

What's the best way to generate an API reference skill from existing web documentation?▼

The best way to generate an API reference skill is running a staged scraper that writes a corpus and navigation map, which a separate skill generator then consumes to produce an interactive queryable reference for developers.

Does the API docs scraper work with any web API documentation site?▼

The API docs scraper processes web API documentation sites by crawling from a provided root URL, extracting queryable endpoint data into a staged corpus that can be consumed by a skill generator regardless of specific site structure.

Related Skills

Explore More Software Engineering Skills→
IhsanDanish25IhsanDanish25
2

theme-detector

Detect and analyze trending market themes across sectors with confidence scores.

Community
Advanced
timscheueraitimscheuerai
24

x-copywriter

Draft X tweets and threads with hooks and precise numbers using curated corpus references.

Community
Intermediate
cajiascajias

q-chat-integration

Delegate specialized tasks to Amazon Q CLI for non-interactive execution.

Community
Advanced
jitenkr2030jitenkr2030

web-reader

Extract structured page data including title, HTML, text, and publication time from URLs.

Community
Intermediate