skills.rest
knoopx
knoopxCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

scraping

Fetch web pages, parse HTML with CSS selectors, and query REST APIs.

68|1|Updated Jan 14, 2026
Search Tags:#automation#web#data-extraction#scrape#apis#css-selectors#nu-shell
One-click install
npx skills add https://github.com/knoopx/pi --skill scraping
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scraping
Source: https://github.com/knoopx/pi/tree/main/agent/skills/scraping
Command: npx skills add https://github.com/knoopx/pi --skill scraping

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web scraping often requires manual steps across multiple tools to fetch pages, parse HTML, and extract structured data. This skill provides a cohesive approach to retrieve web content, query APIs, and automate browser-driven scraping.

Core Features & Use Cases

  • Fetch Web Pages: Retrieve HTML content from websites via HTTP requests.
  • Parse HTML with CSS Selectors: Extract text, attributes, and structured data using CSS queries.
  • API Interactions: Query JSON REST APIs and integrate results into pipelines.
  • Dynamic Content: Use browser automation for pages that render with JavaScript.
  • Example: Collect product prices from an e-commerce site and compile results into a CSV.

Quick Start

Use the scraping skill to fetch the title of a page at https://example.com and extract all h1 elements.

Frequently Asked Questions about scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract data from web pages using CSS selectors?▼

To extract data from web pages, the scraping skill uses automated fetching to retrieve HTML content and CSS selectors to query and parse text, attributes, and structured data. It handles HTTP requests and returns the extracted elements directly in your pipeline.

Can I scrape dynamic websites that render content with JavaScript?▼

Yes, you can scrape dynamic websites by using optional browser automation tooling within the scraping skill. This approach handles pages that render content with JavaScript, allowing automated browser-driven scraping across dynamic sites.

Do I need nu-shell to query REST APIs and extract web data?▼

Yes, you need nu-shell installed along with the query web plugin to query REST APIs and extract web data. The scraping skill relies on this environment to fetch HTML, parse selectors, and integrate JSON API results into pipelines.

What is the best way to collect product prices from an e-commerce site?▼

The best way to collect product prices is using the scraping skill to fetch the e-commerce site HTML, extract prices via CSS selectors, and compile the structured results into a CSV format for analysis.

Does this approach support querying JSON REST APIs?▼

Yes, this scraping approach supports querying JSON REST APIs to retrieve data and integrate the results into your pipelines. It works alongside HTML fetching and CSS selector parsing to provide a cohesive data extraction workflow.

Why does web scraping require multiple manual steps across different tools?▼

Web scraping often requires manual steps because fetching pages, parsing HTML, and extracting structured data are typically handled by separate tools. The scraping skill solves this by providing a cohesive approach to retrieve web content and automate browser-driven scraping.

Related Skills

Explore More Data & Analytics Skills→
zhongjiszhongjis

scraping

Fetch web pages, parse HTML with CSS selectors, and query REST APIs.

Community
Intermediate
takemi-ohamatakemi-ohama
1

data-analyst-export

Export query results to CSV, JSON, Excel, and Markdown formats.

Community
Intermediate
openproseopenprose
1.7k

websh

Browse and query web pages as a Unix-like filesystem with cached content.

Official
Intermediate
stephanjstephanj
9

web-to-markdown

Convert JavaScript-rendered web pages into structured Markdown files.

Community
Intermediate