skills.rest
icyzh
icyzhCOMMUNITY
Explore all skills by this author →
Website𝕏@icyzhbeGitHub

scrapling

Scrape static and JavaScript-rendered web pages using stealth browser automation.

1|Updated Jul 31, 2026
Search Tags:#automation#cloudflare#scraping#crawling#spider#stealth
One-click install
npx skills add https://github.com/icyzh/hermes-web --skill scrapling-icyzh
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/icyzh/hermes-web/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/icyzh/hermes-web --skill scrapling-icyzh

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling, playwright.

What problem does it solve?

This skill solves the challenge of gathering data from modern, protected websites that employ anti-bot measures, Cloudflare challenges, or dynamic JavaScript rendering.

Core Features & Use Cases

  • Stealth Browsing: Bypasses sophisticated bot detection and Cloudflare Turnstile challenges using advanced browser impersonation.
  • Dynamic Content Handling: Executes JavaScript to scrape Single Page Applications (SPAs) and lazy-loaded content.
  • Spider Framework: Enables multi-page crawling with link following, session management, and checkpointing for large-scale data collection.

Quick Start

Use the scrapling skill to extract content from the target URL using stealth mode to bypass bot protection.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape dynamic JavaScript-rendered pages protected by Cloudflare?▼

Stealth browser automation executes JavaScript to render dynamic content while impersonating real browsers to bypass Cloudflare challenges. It handles anti-bot detection to successfully extract data from protected single-page applications.

What is the best way to crawl multiple pages with link following for research?▼

A spider framework enables multi-page crawling by following links and managing sessions for large-scale data collection. It supports checkpointing to track progress and ensure comprehensive research extraction across connected web pages.

Do I need Python 3.10 and playwright installed to use stealth web scraping?▼

Yes, stealth web scraping requires Python 3.10 or higher with the scrapling library and playwright fetcher dependencies installed. These provide the necessary browser automation environment to execute JavaScript and bypass anti-bot measures.

Does stealth scraping work on static HTML pages or only dynamic content?▼

Stealth scraping works on both static HTML and dynamic JavaScript-rendered pages. It handles lazy-loaded content and single-page applications while also supporting standard static content extraction for comprehensive data collection.

Why does my web scraping fail on websites with sophisticated bot detection?▼

Web scraping fails on sophisticated bot detection because standard automated requests lack valid browser fingerprints and JavaScript execution capabilities. Advanced browser impersonation is required to bypass these anti-bot measures and access protected content.

Related Skills

Explore More Education & Research Skills→
Chris-Chai-MinjaeChris-Chai-Minjae

scrapling

Automate web scraping with HTTP fetching, stealth browser automation, and spider crawling.

Community
Advanced
bendruckerbendrucker
16

json

Process JSON data with jq filters for parsing, filtering, and transforming.

Community
Basic
SriRamkunamsettySriRamkunamsetty

scrapling

Fetch web content via HTTP, dynamic JS, and stealth strategies.

Community
Advanced
gyuhagyuha
2

scrapling-official

Scrape and crawl websites with HTTP requests, browser automation, and anti-bot bypass.

Community
Advanced