skills.rest
mupengi-bot
mupengi-botCOMMUNITY
Explore all skills by this author →
GitHub

data-scraper

Fetch HTML from URLs and extract structured data into JSON or Markdown.

11|2|Updated Feb 6, 2026
Search Tags:#automation#data extraction#data collection#web scraping#web crawling#html parsing
One-click install
npx skills add https://github.com/mupengi-bot/mupengism --skill data-scraper
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper
Source: https://github.com/mupengi-bot/mupengism/tree/main/skills/data-scraper
Command: npx skills add https://github.com/mupengi-bot/mupengism --skill data-scraper

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the process of gathering information from websites, transforming raw web data into structured, actionable formats for analysis and monitoring.

Core Features & Use Cases

  • Web Data Collection: Automatically fetches HTML content from specified URLs.
  • Text Extraction: Extracts plain text from HTML, making it easier to process.
  • Structured Data Output: Parses extracted data into JSON or Markdown tables for various applications like competitive analysis, price tracking, and job monitoring.
  • Use Case: Automatically collect product prices from an e-commerce site daily to track price fluctuations and identify sales opportunities.

Quick Start

Use the data-scraper skill to collect pricing information from the given URL.

Frequently Asked Questions about data-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate web scraping to extract structured data from web pages?▼

Web scraping automation fetches HTML content from URLs and extracts structured data into JSON or Markdown formats. This skill uses curl for fetching content and provides options for text or HTML output to streamline data collection.

Can I use HTML parsing to collect product prices for competitive analysis?▼

HTML parsing collects product prices by extracting plain text from fetched HTML content. The extracted data is parsed into structured JSON or Markdown tables, making it easier to track price fluctuations and identify sales opportunities.

What is the best way to convert extracted web content into JSON or Markdown formats?▼

Converting extracted web content into JSON or Markdown formats is done by parsing fetched HTML and transforming the raw data into structured tables. Results are saved to memory and events are generated for downstream processing.

Does web crawling require any external dependencies to fetch and parse HTML content?▼

Web crawling with this skill requires no external dependencies to fetch and parse HTML content. It utilizes curl internally to fetch URLs and extracts plain text or HTML output autonomously.

What are the limitations of using curl for web data extraction from dynamic websites?▼

Using curl for web data extraction limits fetching to static HTML content returned by the server. It does not execute JavaScript, so dynamically rendered content or interactive elements on web pages will not be captured.

When do I need web data collection for trend monitoring and job posting aggregation?▼

Web data collection is needed for trend monitoring and job posting aggregation when you must automatically gather and structure information from various websites. It transforms raw HTML into actionable formats for continuous analysis.

Related Skills

Explore More Data & Analytics Skills→
vibewatchvibewatch

fetch-url

Fetch URLs and extract readable text from HTML pages and PDFs.

Official
Intermediate
cyyehcyyeh
1

data-exploration

Profile datasets to summarize structure, quality, distributions, and anomalies.

Community
Intermediate
caoqiubozhangchenqin2caoqiubozhangchenqin2

smart-web-fetch

Extract clean Markdown content from URLs using fallback services.

Community
Intermediate
eddmanneddmann
2

playwright

Render dynamic web pages with Playwright to capture HTML, text, and screenshots.

Community
Intermediate