webclaw

Extract text, metadata, and structured data from web pages into Markdown and JSON.

2.1k|213|Updated Mar 10, 2026

Search Tags:#data extraction #markdown #json #web scraping #content formatting

Ready to run

npx skills add https://github.com/0xMassi/webclaw --skill webclaw-0xmassi

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: webclaw
Source: https://github.com/0xMassi/webclaw/tree/main
Command: npx skills add https://github.com/0xMassi/webclaw --skill webclaw-0xmassi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires reqwest, wreq, serde, serde_json, tokio, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill turns websites into clean, machine-readable content, making it easier for AI systems and humans to understand and process the information.

Core Features & Use Cases

Web Extraction: Extracts text, metadata, and structured data from websites.
Content Formatting: Converts HTML content into Markdown, JSON, and other formats suitable for AI processing.
Use Case: Use this Skill to quickly create a structured, machine-readable version of a web page for use in an AI-powered document summarizer or search engine.

Quick Start

Extract the content from the URL 'https://example.com' and format it as Markdown using the webclaw skill.

Frequently Asked Questions about webclaw

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I convert website HTML content into Markdown for AI processing?▼

This Skill extracts text, metadata, and structured data from web pages, converting raw HTML into clean Markdown and JSON formats suitable for AI processing and human consumption. It handles HTTP requests and parsing to deliver structured content.

Can I extract structured JSON data from a web page using an HTTP client?▼

Yes, this Skill uses HTTP client capabilities and parsing libraries to retrieve web pages and convert HTML content into structured JSON. It transforms extracted text and metadata into machine-readable formats for data retrieval applications.

What's the best way to scrape web page metadata and text for a document summarizer?▼

The best way is to use a web extraction tool that converts HTML into structured Markdown or JSON. This provides clean, machine-readable content without HTML clutter, making it easier for AI systems to process and summarize information.

Do I need tokio and reqwest to extract structured data from websites in Rust?▼

Yes, this Skill requires HTTP client capabilities and parsing libraries, using dependencies including reqwest, wreq, serde, serde_json, and tokio. These handle asynchronous web requests and data serialization for extracting structured data from websites.

Why does my web scraping output contain unformatted HTML instead of clean text?▼

Your web scraping output contains unformatted HTML because the extraction process lacks a content formatting step. You need a tool that parses HTML and converts it into Markdown or JSON to get clean, machine-readable content for AI systems.