web-reader

Name: web-reader
Availability: InStock
Author: skkarki

Extract web page content and metadata via CLI or SDK.

Updated Aug 1, 2024

Search Tags:#backend #HTML #metadata #web #web-scraping #content-extraction

Ready to run

npx skills add https://github.com/skkarki/Sanam --skill web-reader-skkarki

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-reader
Source: https://github.com/skkarki/Sanam/tree/main/skills/web-reader
Command: npx skills add https://github.com/skkarki/Sanam --skill web-reader-skkarki

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires z-ai-web-dev-sdk, and includes scripts (resource) components.

What problem does it solve?

This skill enables automated extraction of content and metadata from web pages for downstream processing, saving manual scraping time and ensuring consistent data structures.

Core Features & Use Cases

Web Content Extraction: Retrieve the main HTML, plain text, and essential metadata from any URL.
Metadata Retrieval: Access publish date, title, and source URL to power search, indexing, and content curation.
Use Case: Build a simple web content monitor that fetches articles and feeds updates to your dashboard.

Quick Start

Use the web-reader skill to fetch and inspect the content of https://example.com/article and print the title, URL, and publish time.

Frequently Asked Questions about web-reader

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract web page content and metadata from a URL?▼

Web content extraction works by fetching a URL via a backend SDK to return structured data including the title, HTML, published time, and usage tokens. This provides a consistent output format for indexing and downstream processing.

Can I retrieve the publish date and title from news sites and blogs?▼

Yes, metadata retrieval from news sites and blogs returns the title, source URL, and publishedTime. This allows you to power search indexing and content curation workflows.

Does web scraping with z-ai-web-dev-sdk support CLI invocation?▼

Yes, web scraping with z-ai-web-dev-sdk supports both CLI and SDK invocation for fetching pages. You get structured outputs including title, URL, and HTML whether running scripts or using the backend directly.

What is the best way to automate article summarization from web pages?▼

The best way to automate article summarization is to fetch plain text and metadata via a content extraction backend. This returns clean structured data ready for downstream summarization pipelines.

How do I get plain text and HTML from documentation sites?▼

To get plain text and HTML from documentation sites, invoke a web reader via CLI or SDK to fetch the target URL. It returns the main HTML content, title, and source URL for indexing.

What limitations should I expect when scraping web content for a dashboard?▼

When scraping web content for a dashboard, limitations depend on the target site's structure and access restrictions. The extraction returns available metadata and HTML, but heavily dynamic pages may restrict backend fetching.