fetching-web-content
CommunityFetch pages and get clean markdown fast
Software Engineering#web scraping#content extraction#markdown generation#python api#anti-bot bypass#stealth browser#SSRF protection
Authorleba01
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Fetching web pages for reading or scraping often returns messy HTML packed with navigation, ads, boilerplate, and sometimes blocked-by-bot responses instead of usable content.
Core Features & Use Cases
- Clean LLM-ready markdown output: Strips navigation, ads, and boilerplate into readable markdown.
- Automatic anti-bot bypass: Detects common blocks (e.g., Cloudflare/DataDome/PerimeterX/Akamai) and escalates from HTTP fetching to a stealth browser when needed.
- Optional metadata extraction: Produces a
FetchResultwith markdown plus page fields like title, author, date, description, canonical URL, hostname, and sitename. - Built-in SSRF protection: Validates URLs to reject private/loopback destinations.
Quick Start
Use the fetching-web-content skill to fetch https://en.wikipedia.org/wiki/Web_scraping and return the page as cleaned markdown.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: fetching-web-content Download link: https://github.com/leba01/stealthfetch/archive/main.zip#fetching-web-content Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.