skills.rest
dalehurley
dalehurleyCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-scraper

Fetch URL content and extract readable text as Markdown, Text, or HTML.

2|1|Updated Feb 5, 2026
Search Tags:#markdown#fetch#url#text#requests#web-scraping#content-extraction
One-click install
npx skills add https://github.com/dalehurley/phpbot --skill web-scraper-dalehurley
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/dalehurley/phpbot/tree/main/skills/web-scraper
Command: npx skills add https://github.com/dalehurley/phpbot --skill web-scraper-dalehurley

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires trafilatura, requests, and includes scripts (resource) components.

What problem does it solve?

This Skill fetches content from any URL and converts web pages into clean, readable text or Markdown for quick consumption and downstream processing.

Core Features & Use Cases

  • Fetches webpage content and extracts meaningful text for documentation, summaries, or offline reading.
  • Converts content into Markdown or plain text to simplify sharing and integration with other tools.
  • Use Case: Save long articles as clean notes or generate summaries from blog posts for quick reviews.

Quick Start

Fetch a URL and return the cleaned content in Markdown format for immediate review.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract readable text from a webpage URL for offline reading?▼

To extract readable text from a webpage URL, this Skill fetches the content and removes navigation, scripts, and styling. It outputs clean Markdown, plain text, or HTML suitable for offline viewing and documentation.

Can I convert web page content into Markdown format automatically?▼

Yes, you can convert web page content into Markdown automatically. The Skill fetches the specified URL and uses content extraction to transform the HTML into structured Markdown for quick consumption and downstream processing.

Does trafilatura work well for extracting article text from URLs?▼

Trafilatura works well for extracting article text from URLs. Combined with the requests library, it fetches web pages and reliably extracts the main textual content while filtering out non-article elements like sidebars and footers.

What is the best way to archive web content as clean text or HTML?▼

The best way to archive web content as clean text or HTML is using a dedicated content extraction tool. This Skill fetches any URL and outputs saved files in Markdown, text, or HTML formats for content archiving.

Why does my web scraping output include irrelevant page elements?▼

Web scraping output includes irrelevant page elements when using basic fetch methods without content extraction. This Skill uses trafilatura to isolate the main article text, removing boilerplate content like menus and advertisements.

Related Skills

Explore More Software Engineering Skills→
JeffLi1993JeffLi1993
696

seo-audit

Analyze HTML structure, canonicalization, and crawl directives for SEO issues.

Community
Intermediate
smallnestsmallnest
190

markitdown

Convert PDFs, Word, Excel, HTML, and images with OCR into Markdown.

Community
Advanced
jpjednorskijpjednorski

qmd-obsidian

Index and search Obsidian vault notes with qmd keyword and semantic queries.

Community
Advanced
X-School-AcademyX-School-Academy
19

media-html5-video-clip

Render HTML, CSS, and JavaScript animations into MP4 video clips using a headless browser.

Official
Intermediate