skills.rest
GPTtang
GPTtangCOMMUNITY
Explore all skills by this author →
GitHub

clean-content-fetch

Extract and clean main web page content into Markdown.

4|1|Updated Mar 14, 2026
Search Tags:#markdown#web scraping#content extraction#data cleaning#wechat#article scraping
One-click install
npx skills add https://github.com/GPTtang/skill-atlas --skill clean-content-fetch
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clean-content-fetch
Source: https://github.com/GPTtang/skill-atlas/tree/main/skills/devops/clean-content-fetch
Command: npx skills add https://github.com/GPTtang/skill-atlas --skill clean-content-fetch

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling, html2text, curl_cffi, playwright, browserforge, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of extracting clean, readable main content from web pages, especially when standard fetching methods are hindered by excessive noise, dynamic rendering, or complex page structures. It is particularly effective for articles, blog posts, news, announcements, and WeChat official account articles.

Core Features & Use Cases

  • Web Content Extraction: Fetches the primary text content from a given URL.
  • Content Cleaning & Noise Reduction: Removes extraneous elements like ads, navigation, and footers to isolate the core message.
  • Markdown Output: Converts the extracted content into Markdown format for easy readability and further processing.
  • WeChat Article Capture: Specifically handles the extraction of content from WeChat official account articles.
  • Dynamic Page Handling: Utilizes tools like Playwright and Browserforge to manage pages with JavaScript-heavy content or complex rendering.

Quick Start

Use the clean-content-fetch skill to get the main article text from the url https://example.com/article.

Frequently Asked Questions about clean-content-fetch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract clean text from a web page and convert it to Markdown?▼

To extract WeChat official account articles as Markdown, use this Skill to fetch the article URL and clean the HTML structure. It specifically targets WeChat content to remove platform noise and output the main text.

What is the best way to scrape dynamically rendered web pages?▼

Yes, you can remove ads and navigation noise during web scraping. This Skill performs content cleaning and noise reduction to isolate the core message, stripping away extraneous elements like footers and navigation bars from the fetched HTML.

Do I need Playwright and scrapling to extract web content?▼

To extract clean content from WeChat official account articles, supply the article URL to this Skill. It handles the specific page structure of WeChat, cleans extraneous platform elements, and converts the main article text into Markdown format.

What are the limitations of using curl_cffi for content extraction?▼

To handle JavaScript-heavy pages during web scraping, this Skill utilizes Playwright and Browserforge. These tools manage complex rendering and dynamic content, ensuring the primary text is accurately fetched and converted to Markdown.

Related Skills

Explore More Content & Communication Skills→
Chris-Chai-MinjaeChris-Chai-Minjae

scrapling

Automate web scraping with HTTP fetching, stealth browser automation, and spider crawling.

Community
Advanced
eykdeykd

web-scraper

Convert web pages to Markdown using the url-to-markdown-cli-tool.

Community
Intermediate
mshaheerzmshaheerz
1

publish-x-article

Convert Markdown files to X Articles drafts with image insertion.

Community
Advanced
danielscholldanielscholl

web

Fetch web pages and convert HTML to clean Markdown.

Community
Intermediate