skills.rest
involvex
involvexCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-scraper

Extract structured data from web pages into JSON, CSV, or Markdown.

Updated Mar 17, 2026
Search Tags:#automation#pagination#data-extraction#web-scraping#structured-data#json-ld
One-click install
npx skills add https://github.com/involvex/tt2-build-wizard --skill web-scraper-involvex
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/involvex/tt2-build-wizard/tree/main/.gemini/skills/web-scraper
Command: npx skills add https://github.com/involvex/tt2-build-wizard --skill web-scraper-involvex

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Automates extracting structured data from web pages, turning tables, lists, product data, and other content into usable formats while managing pagination and monitoring changes.

Core Features & Use Cases

  • Multi-strategy extraction: static WebFetch, Browser automation for JS-rendered pages, and Bash/API endpoints for diverse sites.
  • Output in CSV/JSON/Markdown with pagination support and diff capabilities.
  • Use Case: Collect product listings across multiple sites and export to CSV for analysis.

Quick Start

Provide a target URL and the data you want to extract; the skill will fetch, parse, and return structured results.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from JavaScript-rendered web pages?▼

To extract structured data from JavaScript-rendered web pages, this skill uses browser automation to execute and capture dynamic content. It handles JS-rendered sites alongside static pages to reliably pull tables, lists, and product data.

Can I scrape product listings across multiple sites and export to CSV?▼

Yes, you can scrape product listings across multiple sites and export to CSV. The skill supports multi-source collections, cross-site comparison, and outputs structured data in JSON, CSV, or Markdown formats.

How does pagination work when scraping structured data from lists?▼

Pagination for scraping structured data is handled automatically within the extraction workflow. The skill navigates through paginated lists to ensure complete data collection before transforming and validating the final output.

What is the best way to automate web scraping without writing custom parsers?▼

The best way to automate web scraping without custom parsers is using a multi-strategy approach. This skill parameterizes workflows from reconnaissance to extraction, identifying tables and lists automatically to output structured JSON.

Does this web scraping approach work with static HTML and API endpoints?▼

Yes, this web scraping approach works with static HTML and API endpoints. It employs a multi-strategy extraction method using WebFetch for static pages and Bash/API endpoints to gather structured data from diverse sites.

Why does my structured data extraction fail on complex dynamic sites?▼

Structured data extraction on complex dynamic sites can fail due to rendering blocks or layout changes. The skill provides robust error handling and provenance metadata to identify extraction failures and validate results.

Related Skills

Explore More Data & Analytics Skills→
dreadnodedreadnode
11

reconnaissance-planning

Plan phased low-to-high noise reconnaissance workflows for external attack surface management.

Official
Intermediate
alexhegitalexhegit
4

pandas-construction-analysis

Transform construction BIM datasets into quantity and cost summaries with Pandas.

Community
Intermediate
Z43LZ43L

scrapling

Extract web content from static, dynamic, and Cloudflare-protected pages.

Community
Advanced
AmidVoshakulAmidVoshakul

web-scraper

Extract structured data from web pages into JSON, CSV, or Markdown.

Community
Advanced