web-scraper

Crawl websites and extract structured data using CSS selectors, JavaScript rendering, and LLM extraction.

45|50|Updated Jan 24, 2026

Search Tags:#data extraction #web scraping #crawl #css selector #html parsing #llm extraction

Ready to run

npx skills add https://github.com/zocomputer/skills --skill web-scraper-zocomputer

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/zocomputer/skills/tree/main/Community/web-scraper
Command: npx skills add https://github.com/zocomputer/skills --skill web-scraper-zocomputer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires httpx, beautifulsoup4, crawl4ai, playwright, and includes scripts (resource) components.

What problem does it solve?

This Skill automates the process of extracting specific data or entire content from websites, overcoming challenges with dynamic content and complex structures.

Core Features & Use Cases

Targeted Data Extraction: Extract specific elements using CSS selectors or predefined schemas.
Full Page Scraping: Fetch and convert entire web pages into Markdown.
JavaScript Rendering: Handles Single Page Applications (SPAs) and dynamic content.
Deep Crawling: Follows links to crawl multiple pages within a site.
LLM-Powered Extraction: Uses AI to extract unstructured data based on natural language instructions.
Batch Processing: Scrape multiple URLs from a file efficiently.
Use Case: Extract product names, prices, and URLs from an e-commerce category page, or crawl a documentation site to gather all articles into a searchable format.

Quick Start

Use the web-scraper skill to scrape the content of https://example.com and save it as markdown.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract structured data from websites with dynamic JavaScript content?▼

Web scraping dynamic content involves rendering JavaScript via Playwright before parsing the HTML structure. This skill handles Single Page Applications by executing scripts and waiting for dynamic elements to load before applying CSS selectors or LLM extraction.

Can I use natural language instructions for LLM-based extraction instead of CSS selectors?▼

LLM extraction allows you to pull unstructured data from web pages using natural language instructions instead of predefined CSS selectors. This skill uses AI to interpret page content and extract specific information based on your descriptive prompts.

What is the best way to crawl an entire documentation site into markdown?▼

Deep crawling a documentation site involves following internal links to fetch multiple pages and converting the HTML content into Markdown. This skill supports deep crawling to systematically extract and compile entire documentation sites into searchable Markdown files.

Does this web scraping tool support batch processing multiple URLs?▼

Batch processing multiple URLs allows you to scrape a list of links from a file efficiently. This skill automates fetching and extracting structured data across numerous URLs simultaneously, streamlining large-scale data extraction and full page scraping tasks.

How does HTML parsing work when handling complex data extraction scenarios?▼

HTML parsing uses Beautiful Soup to navigate the document tree and apply CSS selectors for targeted data extraction. For complex scenarios, this skill escalates to JavaScript rendering or LLM-based extraction to handle dynamic content and unstructured data.