web-content-fetcher

Extracts main article content from URLs into clean Markdown.

1|1|Updated Mar 25, 2026

Search Tags:#markdown #wechat #web-scraping #web-content #content-extraction #scrapling

Ready to run

npx skills add https://github.com/penghang1223/niannian-workspace --skill web-content-fetcher-penghang1223

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-content-fetcher
Source: https://github.com/penghang1223/niannian-workspace/tree/main/skills/web-content-fetcher-hanya
Command: npx skills add https://github.com/penghang1223/niannian-workspace --skill web-content-fetcher-penghang1223

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling, html2text, and includes scripts (resource) components.

What problem does it solve?

Automates turning web articles and pages into clean Markdown, saving manual copy/paste and formatting time.

Core Features & Use Cases

Primary scraping using Scrapling with auto fast → stealth fallback for JS-heavy sites
Secondary option: Jina Reader for simple pages
Preserves headings, links, images, lists, and code blocks
Zero-configuration: works out of the box for blogs, news, documentation, and platforms like 微信公众号
Trigger phrases: "帮我读一下这篇文章", "抓取这个网页", "提取正文", or "read this page for me"

Quick Start

Tell me a URL and I'll fetch and return its main content as clean Markdown.

Frequently Asked Questions about web-content-fetcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract main article content from a URL into clean Markdown?▼

You can extract web content into Markdown by providing a URL to this fetcher. It uses Scrapling to retrieve the page, preserving headings, links, images, and code blocks in the final Markdown output.

Does web scraping work with JavaScript-heavy sites and WeChat Official Accounts?▼

Yes, scraping works with JavaScript-heavy sites and WeChat Official Accounts. It uses Scrapling with an automatic stealth fallback to render dynamic content and extract the article into Markdown.

What web elements are preserved when converting web pages to Markdown?▼

Converting web pages to Markdown preserves essential structural elements like headings, links, images, lists, and code blocks. This ensures the extracted Markdown accurately reflects the original web page layout.

How does Scrapling handle content extraction when a fast fetch fails?▼

When fast fetching fails, Scrapling handles content extraction by falling back to a stealth mode for dynamic rendering. A secondary Jina Reader option is also available to parse simpler web pages.

Do I need any configuration to scrape blogs and documentation pages into Markdown?▼

No configuration is needed to scrape blogs and documentation pages into Markdown. The tool works out of the box, automatically applying fast fetching or stealth fallback to extract the main article content.