wikipedia

Scrape Wikipedia articles and extract text, headings, references, and categories.

5|Updated Apr 9, 2026

Search Tags:#html #api #scraping #content-extraction #cheerio #wikipedia

Ready to run

npx skills add https://github.com/AgentComputerAI/torch --skill wikipedia-agentcomputerai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: wikipedia
Source: https://github.com/AgentComputerAI/torch/tree/main/skills/sites/wikipedia
Command: npx skills add https://github.com/AgentComputerAI/torch --skill wikipedia-agentcomputerai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Wikipedia pages contain valuable information but extracting the full text, headings, references, and categories requires handling changing HTML structures and rate‑limit policies. This Skill provides a reliable way to obtain clean, structured article data without dealing with anti‑bot measures.

Core Features & Use Cases

HTML scraping with cheerio that correctly handles the new heading‑wrapper <div class="mw-heading …"> introduced in 2023.
Automatic reference and category extraction giving complete citation lists and topic tags.
Fallback to Wikipedia’s REST and Action APIs for faster structured data when HTML is not needed.
Use case: Gather the full content and metadata of a set of Wikipedia articles for a research dataset or knowledge‑base population.

Quick Start

Ask the Wikipedia skill to retrieve the full text, sections, references, and categories of the article “Web scraping”.

Frequently Asked Questions about wikipedia

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape Wikipedia articles into structured data?▼

Scrape Wikipedia articles into structured data by fetching the HTML with a descriptive User-Agent and parsing it with cheerio to extract full text, headings, references, and categories. This handles the new heading-wrapper divs for clean content extraction.

How does cheerio handle the new Wikipedia heading wrapper divs during content extraction?▼

Cheerio handles Wikipedia heading wrapper divs by specifically targeting the `mw-heading` class introduced in 2023, allowing accurate traversal and extraction of section titles and full article text without breaking on updated HTML structures.

What is the best way to extract Wikipedia references and categories without triggering anti-bot measures?▼

The best way to extract references and categories without triggering anti-bot measures is using a descriptive User-Agent for HTTP fetching, or falling back to Wikipedia's REST and Action APIs for faster structured data retrieval.

Can I use the Wikipedia API instead of HTML scraping to get article text?▼

Yes, you can use the Wikipedia API instead of HTML scraping. The skill provides a fallback to Wikipedia's REST and Action APIs to retrieve faster structured data when full HTML parsing with cheerio is not needed.

Does Wikipedia scraping work with language wikis other than en.wikipedia.org?▼

Yes, Wikipedia scraping works with language wikis other than en.wikipedia.org. The skill applies to retrieving full article text, headings, references, and categories for any language wiki page hosted by Wikipedia.

Why does my Wikipedia scraper fail to extract headings after the 2023 HTML update?▼

Wikipedia scrapers fail to extract headings after the 2023 HTML update because the page structure changed to use heading wrapper `<div class="mw-heading">` elements, which requires updated cheerio parsing logic to handle correctly.