skills.rest
XPRNetwork
XPRNetworkOFFICIAL
Explore all skills by this author →
Website𝕏@XPRNetworkGitHub

web-scraping

Fetch web pages and extract clean text, titles, and links.

3|3|Updated Feb 5, 2026
Search Tags:#markdown#html#fetch#data-extraction#web-scraping#link-extraction
One-click install
npx skills add https://github.com/XPRNetwork/xpr-agents --skill web-scraping-xprnetwork
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraping
Source: https://github.com/XPRNetwork/xpr-agents/tree/main/openclaw/starter/agent/skills/web-scraping
Command: npx skills add https://github.com/XPRNetwork/xpr-agents --skill web-scraping-xprnetwork

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web scraping tools for fetching and extracting data from web pages enable rapid gathering of text, metadata, and links without manual copy-paste, saving time and reducing errors.

Core Features & Use Cases

  • Fetch a single page and return cleaned text, title, description, and link count in a chosen format (text, markdown, or html).
  • Discover and classify links as internal or external with optional filtering, for research, data collection, or competitive analysis.
  • Fetch multiple pages in parallel to compare content and extract consistent data across sources.

Quick Start

Ask your agent to fetch a page using scrape_url and return its text content in markdown for quick review.

Frequently Asked Questions about web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fetch a web page and extract clean text content for analysis?▼

Web scraping fetches a single page to extract cleaned text, titles, descriptions, and link counts in markdown, text, or html format. You simply provide the URL and chosen output format for analysis-ready content.

What is the best way to extract all links from an HTML page and classify them?▼

Web scraping extracts all links with text and type from fetched HTML pages, classifying them as internal or external with optional filtering. This yields structured link data for research, data collection, or competitive analysis.

Can I fetch multiple web pages in parallel to compare content across sources?▼

Web scraping fetches multiple pages in parallel to compare content and extract consistent data across sources. This enables side-by-side analysis of text and metadata from different web pages simultaneously.

What are the size and timeout limits for fetching web pages?▼

Web scraping enforces a 5MB content limit and a 30-second timeout per fetch. Safe HTML parsing prevents large or unsafe responses, ensuring reliable data extraction without overwhelming system resources.

Does web scraping require external dependencies or libraries to fetch pages?▼

Web scraping requires no external dependencies or libraries to fetch pages. It handles safe HTML parsing and data extraction internally, returning clean text and links directly through your agent.

Related Skills

Explore More Data & Analytics Skills→
guyruguyru

ddg-search

Search the web via DuckDuckGo HTML and output titles, URLs, and snippets.

Community
Intermediate
diegosouzapwdiegosouzapw
44.0k

omniroute-web-fetch

Fetch webpages and convert them to LLM-ready markdown via OmniRoute.

Community
Basic
Brilly-BohyunBrilly-Bohyun

sketch

Generate interactive HTML mockups for comparing 2-3 design variants.

Community
Intermediate
johnlindquistjohnlindquist
24

spider

Crawls websites with HTTP fetch, link extraction, and HTML parsing for JSON output.

Community
Intermediate