content-extract

Convert URLs into Markdown with traceable source metadata.

55|5|Updated Feb 10, 2026
One-click install
npx skills add https://github.com/blessonism/openclaw-skills --skill content-extract
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: content-extract
Source: https://github.com/blessonism/openclaw-skills/tree/main/content-extract
Command: npx skills add https://github.com/blessonism/openclaw-skills --skill content-extract

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This skill centralizes the transformation of a URL into a readable, Markdown-formatted document with a traceable source contract, enabling downstream OpenClaw workflows to rely on consistent inputs.

Core Features & Use Cases

  • Deterministic extraction using a cheap probe via web_fetch first, with MinerU as a fallback for anti-bot or dynamic pages.
  • GitHub fast path for repository READMEs and metadata, avoiding unreliable client-side rendering.
  • Unified Result Contract output including source_url, engine, markdown, artifacts, and sources, ensuring traceability.

Quick Start

Convert the given URL into clean, Markdown-formatted output with traceable source links.

Frequently Asked Questions about content-extract

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a URL to Markdown for downstream analysis?

To convert a URL to Markdown, the skill first probes the page via web_fetch. If the site blocks the probe or renders dynamically, it falls back to MinerU, returning a clean Markdown document with traceable source links for analysis.

Can I extract README and metadata directly from a GitHub repository URL?

Yes, you can extract README and metadata from a GitHub repository URL using a dedicated fast path. This bypasses unreliable client-side rendering to directly fetch repository metadata and return clean, Markdown-formatted content.

What is the best way to extract content from anti-bot or dynamic web pages?

The best way to extract content from anti-bot or dynamic web pages is using MinerU as a fallback. The extraction process initially attempts a cheap web_fetch probe, then automatically switches to MinerU for sites that block the initial probe.

Does web content extraction support a domain whitelist to skip probing?

Yes, web content extraction supports a domain whitelist feature. You can configure specific domains to skip the initial web_fetch probing step, streamlining the extraction process for trusted sources and returning Markdown output faster.

Why does URL content extraction return a unified Result Contract with source traceability?

URL content extraction returns a unified Result Contract with source traceability to ensure downstream workflows receive consistent inputs. It includes fields like source_url, engine, markdown, artifacts, and sources, guaranteeing reliable data lineage.

What are the limitations of using web_fetch for URL to Markdown conversion?

The limitation of using web_fetch for URL to Markdown conversion is its inability to bypass anti-bot mechanisms or render dynamic content. When web_fetch fails on such sites, the extraction automatically falls back to MinerU.