web-fetch

Name: web-fetch
Availability: InStock
Author: zhayujie

Fetch web page HTML and extract title and readable text.

46.4k|10.3k|Updated Aug 7, 2022

Search Tags:#HTML #fetch #curl #text-extraction #title #web-page

Ready to run

npx skills add https://github.com/zhayujie/chatgpt-on-wechat --skill web-fetch-zhayujie

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-fetch
Source: https://github.com/zhayujie/chatgpt-on-wechat/tree/main/skills/web-fetch
Command: npx skills add https://github.com/zhayujie/chatgpt-on-wechat --skill web-fetch-zhayujie

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires curl, and includes scripts (resource) components.

What problem does it solve?

Fetch web page content by downloading HTML and extracting the title and readable text.

Core Features & Use Cases

curl-based HTTP requests to fetch HTML content without a browser
Title and text extraction with basic HTML stripping
Output to stdout or optional file for easy integration

Quick Start

bash "<base_dir>/scripts/fetch.sh" "https://example.com"

Frequently Asked Questions about web-fetch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract readable text and title from HTML without a browser?▼

Extracting readable text and title from HTML without a browser involves downloading the web page via curl and stripping HTML tags. This lightweight method retrieves content for quick previews or archiving without requiring browser automation.

What is the best way to fetch web page content for quick previews?▼

Fetching web page content for quick previews is best done using curl-based HTTP requests to download HTML and extract the title and readable text. The result outputs to stdout or a file for easy integration.

Do I need curl to extract text from arbitrary URLs?▼

Yes, curl is required to extract text from arbitrary URLs using this approach. Curl performs the HTTP requests to fetch HTML content, which is then parsed to extract the title and readable text.

Can I save extracted web page text to a file?▼

Yes, you can save extracted web page text to a file. The script outputs the extracted title and readable text to stdout by default, or directs the result to an optional file for easy integration and archiving.

What are the limitations of basic HTML stripping for text extraction?▼

Limitations of basic HTML stripping for text extraction include the inability to render dynamic JavaScript content, as it lacks browser automation. It targets lightweight retrieval from arbitrary URLs, focusing solely on basic title and readable text extraction.