web-fetch

Fetch web page HTML and extract title and readable text.

46.4k|10.3k|Updated Aug 7, 2022
One-click install
npx skills add https://github.com/zhayujie/chatgpt-on-wechat --skill web-fetch-zhayujie
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: web-fetch
Source: https://github.com/zhayujie/chatgpt-on-wechat/tree/main/skills/web-fetch
Command: npx skills add https://github.com/zhayujie/chatgpt-on-wechat --skill web-fetch-zhayujie

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires curl, and includes scripts (resource) components.

What problem does it solve?

Fetch web page content by downloading HTML and extracting the title and readable text.

Core Features & Use Cases

  • curl-based HTTP requests to fetch HTML content without a browser
  • Title and text extraction with basic HTML stripping
  • Output to stdout or optional file for easy integration

Quick Start

bash "<base_dir>/scripts/fetch.sh" "https://example.com"

Frequently Asked Questions about web-fetch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract readable text and title from HTML without a browser?

Extracting readable text and title from HTML without a browser involves downloading the web page via curl and stripping HTML tags. This lightweight method retrieves content for quick previews or archiving without requiring browser automation.

What is the best way to fetch web page content for quick previews?

Fetching web page content for quick previews is best done using curl-based HTTP requests to download HTML and extract the title and readable text. The result outputs to stdout or a file for easy integration.

Do I need curl to extract text from arbitrary URLs?

Yes, curl is required to extract text from arbitrary URLs using this approach. Curl performs the HTTP requests to fetch HTML content, which is then parsed to extract the title and readable text.

Can I save extracted web page text to a file?

Yes, you can save extracted web page text to a file. The script outputs the extracted title and readable text to stdout by default, or directs the result to an optional file for easy integration and archiving.

What are the limitations of basic HTML stripping for text extraction?

Limitations of basic HTML stripping for text extraction include the inability to render dynamic JavaScript content, as it lacks browser automation. It targets lightweight retrieval from arbitrary URLs, focusing solely on basic title and readable text extraction.