skills.rest
awfixers-stuff
awfixers-stuffOFFICIAL
Explore all skills by this author →
Website𝕏@awfixerGitHub

web-fetch

Convert HTTP/HTTPS web pages into clean Markdown using selectors and DOM parsing.

Updated Aug 27, 2026
Search Tags:#markdown#fetch#scraping#web#content-extraction#turndown#html-extraction
One-click install
npx skills add https://github.com/awfixers-stuff/opencode-config --skill web-fetch-awfixers-stuff
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-fetch
Source: https://github.com/awfixers-stuff/opencode-config/tree/main/skills/web-fetch
Command: npx skills add https://github.com/awfixers-stuff/opencode-config --skill web-fetch-awfixers-stuff

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Extracts readable, cleaned Markdown from arbitrary HTTP/HTTPS pages by preferring markdown-native responses and falling back to selector-based HTML extraction and a DOM-to-markdown parser when selectors fail. It eliminates manual copy-paste and noisy HTML by detecting content type, isolating content containers, and stripping navigation and UI artifacts.

Core Features & Use Cases

  • Content-type prioritization: Prefers text/markdown endpoints to preserve native Markdown content.
  • Selector-based extraction: Uses site-specific or generic CSS selectors to extract article/main content and remove nav/header/footer elements.
  • DOM-to-Markdown fallback: Provides a Bun-based parser that selects the largest content region, cleans UI elements, and converts HTML to well-formed Markdown with preserved code blocks.
  • Use Case: Ideal for grabbing documentation pages, blog articles, or reference material for summarization, archiving, or inclusion into knowledge bases.

Quick Start

Fetch https://example.com and convert the page to clean markdown, preferring markdown responses and falling back to selector extraction or the Bun DOM-to-markdown parser.

Frequently Asked Questions about web-fetch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert HTML web pages to clean Markdown?▼

Converting web pages to Markdown involves detecting content types, prioritizing text/markdown responses, and falling back to selector-based HTML extraction. This process removes navigation and UI artifacts, yielding clean Markdown for articles or documentation.

How does web scraping handle documentation pages with code blocks?▼

Scraping documentation pages preserves code blocks and headings by applying a Bun-based Turndown fallback parser. It selects the largest content region, removes UI elements, and converts HTML to well-formed Markdown while maintaining technical formatting.

What is the best way to extract article content and remove navigation elements?▼

Extracting article content while removing navigation is achieved through selector-based inclusion and exclusion. The Skill targets article and main content containers with site-specific or generic CSS selectors, explicitly stripping header, footer, and nav artifacts.

Does content extraction work on unfamiliar sites without specific selectors?▼

Content extraction handles unfamiliar sites by applying generic CSS selectors to isolate main content. If initial selectors fail, a Bun-based DOM-to-markdown parser selects the largest content region and automatically cleans remaining UI elements to produce Markdown.

Why does my fetched web page include UI artifacts and navigation text?▼

Fetched pages include UI artifacts when generic selectors fail to isolate the main content region. The DOM-to-markdown fallback addresses this by selecting the largest content area and cleaning UI elements, though highly non-standard page structures may cause residual noise.

Related Skills

Explore More Content & Communication Skills→
diegosouzapwdiegosouzapw
44.0k

omniroute-web-fetch

Fetch webpages and convert them to LLM-ready markdown via OmniRoute.

Community
Basic
jykimjykim
1

markdown-to-docx

Convert Markdown documents to .docx with Korean font support and inline formatting.

Community
Intermediate
robomotioniorobomotionio
2

web-scraper

Extract structured data from HTML using CSS selectors or XPath queries.

Official
Intermediate
SpaceZephyrSpaceZephyr
333

web-scraper

Fetch URLs and convert HTML page content into readable markdown.

Community
Intermediate