web-fetcher

Convert web pages into clean Markdown via OpenCLI or Jina Reader.

36|1|Updated Mar 24, 2026
One-click install
npx skills add https://github.com/jiahao-shao1/sjh-skills --skill web-fetcher
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: web-fetcher
Source: https://github.com/jiahao-shao1/sjh-skills/tree/main/skills/web-fetcher
Command: npx skills add https://github.com/jiahao-shao1/sjh-skills --skill web-fetcher

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Fetch any URL as clean Markdown. Use instead of WebFetch for JS-rendered pages, login-required platforms (Twitter/X, zhihu, reddit, weibo, xiaohongshu, bilibili, etc.), and complex pages. Routes known platforms through OpenCLI (browser login state), others through Jina Reader / defuddle.md / markdown.new. Invoke when the user provides a URL to read, extract, summarize, or convert to markdown.

Core Features & Use Cases

  • Known platforms via OpenCLI (uses browser login state) for reliable rendering and access control.
  • Generic URLs through a deterministic fallback chain: Jina Reader → defuddle.md → markdown.new.
  • Use Case: convert research articles, blog posts, or documentation pages into Markdown for offline reading or integration into notes.

Quick Start

Fetch a URL and output the cleaned Markdown to stdout or save it to a file.

Frequently Asked Questions about web-fetcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a web page into clean Markdown for documentation?

To convert a web page into clean Markdown, this tool routes generic URLs through a deterministic chain of Jina Reader, defuddle.md, and markdown.new, producing output suitable for documentation and research workflows.

Can I extract Markdown from login-protected platforms like Reddit or Twitter?

Yes, you can extract Markdown from login-protected platforms like Reddit or Twitter by routing requests through OpenCLI, which leverages your existing browser login state to access and render restricted content reliably.

Does web scraping JavaScript-rendered pages require complex setup?

Scraping JavaScript-rendered pages requires minimal setup when using a deterministic fallback chain. It relies on either configured OpenCLI or built-in readers like Jina Reader to fetch and convert dynamic content automatically.

What is the best way to fetch generic URLs when OpenCLI is not configured?

The best way to fetch generic URLs without OpenCLI is using a deterministic fallback chain: Jina Reader attempts first, followed by defuddle.md, and finally markdown.new, ensuring reliable Markdown extraction for most web pages.

Why use a deterministic fetching chain instead of standard WebFetch for web scraping?

A deterministic fetching chain is preferred over standard WebFetch because it successfully handles JavaScript-rendered pages and login-required platforms, producing clean Markdown output where standard fetching methods fail to load dynamic content.