skills.rest
kevinnft
kevinnftCOMMUNITY
Explore all skills by this author →
𝕏@nossxbtGitHub

web-scraping

Extract structured data from static HTML, JS-rendered pages, and hybrid sites.

13|3|Updated May 12, 2026
Search Tags:#data extraction#web scraping#spa#headless browser#javascript-rendered#api discovery
One-click install
npx skills add https://github.com/kevinnft/ai-agent-skills --skill web-scraping-kevinnft
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraping
Source: https://github.com/kevinnft/ai-agent-skills/tree/main/skills/research/web-scraping
Command: npx skills add https://github.com/kevinnft/ai-agent-skills --skill web-scraping-kevinnft

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Enable automated extraction of data from websites, including JavaScript-rendered content, so users can collect structured data without manual copying or scraping from multiple pages.

Core Features & Use Cases

  • SPA data extraction from dynamic pages
  • Headless browser automation to render and interact with pages
  • API discovery and data normalization for downstream analytics
  • Real-world use cases: price tracking, content aggregation, competitive intelligence

Quick Start

Run a headless browser to load a target website and extract structured data from the rendered page.

Frequently Asked Questions about web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract data from JavaScript-rendered SPA pages?▼

To extract data from JavaScript-rendered SPA pages, this capability uses headless browser automation to fully load dynamic content and retrieve structured data from the rendered page.

What is the best way to scrape a website with no public API?▼

The best way to scrape a website with no public API is applying scraping workflows that support API discovery, static HTML extraction, and headless browser automation to capture and normalize data.

Can I use headless browser automation for competitive intelligence gathering?▼

Yes, you can use headless browser automation for competitive intelligence gathering, applying scraping workflows to extract and normalize structured data from dynamic and hybrid sites.

How does data normalization work after web scraping?▼

Data normalization after web scraping works by transforming extracted static HTML and SPA content into structured formats with robust error handling for downstream analytics.

Does web scraping work on hybrid sites with static HTML and dynamic content?▼

Web scraping works on hybrid sites by applying both static HTML extraction and headless browser automation to capture JavaScript-rendered content and normalize the structured output.

Why does my web scraper not working on SPA content need robust error handling?▼

A web scraper not working on SPA content needs robust error handling because dynamic JavaScript rendering introduces asynchronous loading failures that require structured data normalization workflows to manage.

Related Skills

Explore More Data & Analytics Skills→
tusosostusosos

web-scraper

Extract data from web pages using CSS selectors and output JSON or CSV.

Community
Advanced
neuron-oneneuron-one
25

nimble

Extracts structured data from multiple websites into normalized JSON tables.

Community
Intermediate
okwindsokwinds
50

headless-web-viewer

Render JavaScript-heavy webpages and extract visible text with Playwright.

Community
Intermediate
patchy631patchy631
36.9k

brightdata-web-mcp

Fetch live web content and extract structured data via Bright Data MCP.

Community
Intermediate