skills.rest
dloschiavo
dloschiavoCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-scraping

Scrape web content from SPAs and anti-bot protected sites using Playwright.

Updated May 13, 2026
Search Tags:#data extraction#Playwright#web scraping#content validation#anti-bot
One-click install
npx skills add https://github.com/dloschiavo/orca --skill web-scraping-dloschiavo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraping
Source: https://github.com/dloschiavo/orca/tree/main/recipe/web-scraping
Command: npx skills add https://github.com/dloschiavo/orca --skill web-scraping-dloschiavo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires playwright, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of scraping web content, addressing challenges like anti-bot protection, directory listings, and paginated sources.

Core Features & Use Cases

  • URL Discovery: Utilizes sitemap.xml, index/directory pages, and search results for entry points.
  • Anti-bot Evasion: Implements strategies to bypass anti-bot measures on SPAs and other sites.
  • Content Validation: Ensures actual results are present and validates content against expected schema.
  • Use Case: Ideal for extracting data from e-commerce websites, news platforms, or any site with complex navigation.

Quick Start

Start the web-scraping skill to scrape content from a specific URL and save the results to 'scraped_data.json'.

Frequently Asked Questions about web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape web content from sites with anti-bot protection?▼

To scrape web content from sites with anti-bot protection, this skill uses Playwright to render JavaScript and bypass restrictions on SPAs and protected platforms. It ensures data extraction succeeds where standard HTTP requests fail.

What is the best way to extract data from paginated web sources?▼

The best way to extract data from paginated web sources is using URL discovery techniques via sitemap.xml, index pages, and search results. This skill automates navigation across pagination to retrieve and validate structured content.

Can I use Playwright to scrape Single Page Applications?▼

Yes, you can use Playwright to scrape Single Page Applications. This skill utilizes Playwright specifically for JavaScript rendering, allowing it to execute dynamic scripts and extract rendered content from complex SPA architectures.

How do I validate extracted web scraping results against an expected schema?▼

You validate extracted web scraping results by implementing content validation checks against expected schemas. This skill ensures actual results are present and structurally accurate before saving the extracted data to JSON.

Does this web scraping tool discover entry point URLs automatically?▼

Yes, this web scraping tool discovers entry point URLs automatically. It utilizes sitemap.xml, directory index pages, and search results to identify starting points, streamlining extraction for e-commerce and news platforms.

Related Skills

Explore More Data & Analytics Skills→
X-School-AcademyX-School-Academy
19

init-playwright

Initialize and verify Playwright CLI, Google Chrome, and the Playwright MCP Bridge extension.

Official
Intermediate
billydeeii136billydeeii136

seo-schema

Detect, validate, and generate Schema.org JSON-LD structured data on web pages.

Community
Advanced
yasinnertenyasinnerten

test-scraper

Test HTML parsing and data extraction logic against diverse HTML structures.

Community
Basic
otrebuotrebu
4

web-to-markdown

Batch-convert JavaScript-rendered web pages to a timestamped Markdown file.

Community
Advanced