web-scraper

Fetch URLs and convert HTML page content into readable markdown.

333|83|Updated Jan 9, 2026

Search Tags:#web scraping #content extraction #markdown conversion #url fetching #html parsing #article reading

Ready to run

npx skills add https://github.com/SpaceZephyr/myskill --skill web-scraper-spacezephyr

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/SpaceZephyr/myskill/tree/main/web-scraper
Command: npx skills add https://github.com/SpaceZephyr/myskill --skill web-scraper-spacezephyr

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests, and includes scripts (resource) components.

What problem does it solve?

This Skill removes the friction of manually copying content from websites by fetching page text and converting messy HTML into readable markdown.

Core Features & Use Cases

Web Page Fetching: Retrieves content directly from URLs for articles, pages, and other public web sources.
HTML to Markdown Conversion: Cleans page structure into readable markdown while preserving headings, lists, links, and code blocks.
Practical Scenarios: Use it to read online articles, extract page text for research, or work around network-limited fetch failures when a simpler fetch method is unavailable.

Quick Start

Ask the web-scraper skill to fetch the target URL and convert the page into clean markdown text.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I convert web page HTML to clean markdown?▼

To convert web page HTML to clean markdown, you fetch the URL and parse the content. This process extracts readable text while preserving headings, lists, links, and code blocks to produce structured markdown output.

Can I extract article text from URLs when direct browser fetching fails?▼

You can extract article text from URLs by applying HTTP fetching as a fallback. This method retrieves public web page content directly, bypassing network-limited browser fetch failures to reliably capture the target text.

Does this web scraping approach support capturing multiple pages at once?▼

This web scraping approach supports scraping multiple pages by applying HTTP fetching to each target URL. You can capture website text across various public sources sequentially, outputting markdown text for reliable text extraction.

What is the best way to clean messy website HTML into readable text?▼

The best way to clean messy website HTML into readable text is HTML parsing combined with markdown conversion. This removes structural friction and normalizes page content into a clean, readable markdown format.

Do I need the requests library to fetch and parse web page content?▼

Yes, you need the requests library to perform the HTTP fetching required by this task. It serves as the core dependency for retrieving URL content before HTML parsing and markdown conversion can be applied.

How does output truncation handle long articles during content extraction?▼

Output truncation handles long articles during content extraction by limiting the final markdown text length. This ensures reliable text extraction and prevents oversized outputs when scraping extensive web pages.