web-scraper

Name: web-scraper
Availability: InStock
Author: tusosos

Extract data from web pages using CSS selectors and output JSON or CSV.

Updated Apr 3, 2026

Search Tags:#automation #etl #data-extraction #data-pipeline #web-scraping #css-selectors

Ready to run

npx skills add https://github.com/tusosos/manus-knowledge-base --skill web-scraper-tusosos

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/tusosos/manus-knowledge-base/tree/main/skills/web-scraper
Command: npx skills add https://github.com/tusosos/manus-knowledge-base --skill web-scraper-tusosos

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web pages often present data in unstructured HTML, making extraction and ingestion into databases labor-intensive. This Skill automates the collection, normalization, and loading of data from websites into structured formats like JSON or CSV, enabling reliable analytics and data pipelines.

Core Features & Use Cases

HTML parsing and CSS-selector based extraction to gather items such as product names, prices, links, and tables from both static and JS-rendered pages.
Data transformation and deduplication to normalize fields and remove duplicates for robust ETL workflows.
Use Case: Build a scalable scraper that aggregates catalog data across sites and loads it into a data warehouse for analysis.

Quick Start

Ask me to scrape a catalog page and save the results to a CSV file.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract data from HTML pages and convert it to JSON or CSV?▼

To extract data from HTML pages, identify target URLs and specific elements like prices or links, then use CSS selectors to parse the content and emit structured JSON or CSV files for downstream loading.

Can I scrape JavaScript-rendered web pages using CSS selectors?▼

Yes, you can scrape JavaScript-rendered web pages by configuring the crawler to use a headless browser instead of static HTML requests, allowing CSS selectors to extract dynamically generated data elements.

How do I build a data pipeline to scrape and deduplicate product catalogs?▼

Build a data pipeline by fetching catalog pages, applying data transformation and deduplication to normalize fields and remove duplicates, then emitting a clean, load-ready payload for databases.

Does web scraping support pagination for large ETL workflows?▼

Yes, web scraping supports pagination to crawl across multiple pages, extracting and normalizing items sequentially to aggregate catalog data for robust ETL workflows and data warehouse loading.

What is the best way to automate web scraping for analytics and database ingestion?▼

The best way to automate web scraping for analytics is to automate collection and normalization of unstructured HTML into structured formats like JSON or CSV, providing a reliable load-ready payload for databases.

Do I need to handle data transformation before loading scraped web data into a warehouse?▼

You do not need external transformation before loading, because the scraping process handles data transformation and deduplication internally, normalizing fields and removing duplicates to provide a clean payload.