skills.rest
tiankong0101-byte
tiankong0101-byteCOMMUNITY
Explore all skills by this author →
GitHub

news-extractor

Extract Chinese news articles into structured JSON and Markdown.

Updated May 17, 2026
Search Tags:#markdown#json#web scraping#wechat#toutiao#news extraction
One-click install
npx skills add https://github.com/tiankong0101-byte/skills-registry --skill news-extractor-tiankong0101-byte
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: news-extractor
Source: https://github.com/tiankong0101-byte/skills-registry/tree/main/skills/news-extractor
Command: npx skills add https://github.com/tiankong0101-byte/skills-registry --skill news-extractor-tiankong0101-byte

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pydantic, requests, curl_cffi, tenacity, parsel, demjson3, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill removes the manual work of copying and cleaning up news articles from major Chinese news platforms, turning messy web pages into structured, reusable content.

Core Features & Use Cases

  • Multi-platform extraction: Supports WeChat Official Accounts, Toutiao, Netease News, Sohu News, and Tencent News.
  • Structured output: Produces both JSON and Markdown, including title, author, publish time, text, images, and videos.
  • Resilient crawling workflow: Uses platform detection, retry logic, browser-like fetching, and page-specific parsers to handle varied article layouts.
  • Use case: A researcher can paste a news URL and quickly receive a clean article archive for analysis, summarization, or publishing.

Quick Start

Ask the assistant to extract the article from a supported Chinese news URL and return it as JSON and Markdown in the format you choose.

Frequently Asked Questions about news-extractor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract WeChat Official Account articles into structured Markdown and JSON?▼

To extract WeChat articles into Markdown and JSON, you need a scraping workflow that performs platform detection, browser-like HTTP fetching, and HTML parsing to isolate article metadata, text, and media. This approach converts messy web pages into clean structured formats for archival or analysis.

Can I scrape news articles from Toutiao and Netease News without manual cleanup?▼

Yes, you can scrape news articles from Toutiao and Netease News without manual cleanup by using a parser that applies platform detection and page-specific extraction rules. This process automatically isolates the core content, removing surrounding page clutter to output clean text and media.

What is the best way to convert Chinese news web pages into structured JSON?▼

The best way to convert Chinese news web pages into structured JSON is using an extraction workflow that combines resilient HTTP fetching with HTML parsing. This method identifies the target platform, extracts the title and publish time, and structures the text alongside extracted media assets.

Does web scraping for Tencent News and Sohu News require retry logic for failed requests?▼

Web scraping for Tencent News and Sohu News does require retry logic to handle failed requests and varied article layouts. Implementing resilient fetching with retry attempts ensures the HTML payload is successfully retrieved before parsing the article content and media into Markdown.

Why does news extraction fail on certain WeChat or Toutiao article links?▼

News extraction can fail on WeChat or Toutiao links due to anti-bot protections or varied page layouts that break standard parsers. Using browser-like fetching with curl and applying platform-specific parsing rules helps bypass these constraints to successfully extract the article content.

Related Skills

Explore More Content & Communication Skills→
SpaceZephyrSpaceZephyr
333

web-scraper

Fetch URLs and convert HTML page content into readable markdown.

Community
Intermediate
alibabaalibaba
10.5k

pdf-extractor

Extract text, tables, and metadata from PDF documents into JSON.

Official
Intermediate
VanLVanL
1

hackernews-frontpage

Parse Hacker News front page HTML into structured JSON story data.

Community
Basic
NanmiCoderNanmiCoder
60

news-extractor

Extract article content from Chinese news platforms into JSON and Markdown.

Community
Advanced