skills.rest
loteiron
loteironCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

scrapling

Scrape websites using HTTP, dynamic browser, and stealth fetching with Cloudflare bypass.

Updated Sep 10, 2026
Search Tags:#data extraction#python#web scraping#crawling#scrapling#cloudflare bypass#stealth browser
One-click install
npx skills add https://github.com/loteiron/ZeusAgent --skill scrapling-loteiron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/loteiron/ZeusAgent/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/loteiron/ZeusAgent --skill scrapling-loteiron

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering or are protected by anti-bot systems like Cloudflare Turnstile, leaving standard HTTP requests and basic extraction tools unable to retrieve the content. ## Core Features & Use Cases - Three Fetching Strategies: Choose between Fetcher for fast static HTTP requests, DynamicFetcher for JS-rendered pages, and StealthyFetcher for Cloudflare-protected sites. - Spider Framework: Crawl multiple pages with link following, concurrent requests, multi-session routing, and pause/resume checkpoints. - CLI and Python API: Extract pages directly from the command line into HTML, Markdown, text, or JSON, or write Python scripts with CSS, XPath, and regex element selection. - Use Case: Imagine you need to collect product listings from a site behind Cloudflare. Use StealthyFetcher with solve_cloudflare enabled to bypass the challenge, then select prices and titles with CSS selectors and export results to JSON. ## Quick Start Use the scrapling skill to scrape the quotes from https://quotes.toscrape.com and save them to a JSON file.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a Cloudflare-protected website in Python?▼

Use Scrapling's StealthyFetcher with solve_cloudflare=True to bypass Cloudflare Turnstile challenges. Options like block_webrtc and hide_canvas reduce browser fingerprinting, though solving adds 5-15 seconds per fetch.

How to scrape JavaScript-rendered pages with Python?▼

Use DynamicFetcher to load pages in a real headless browser that executes JavaScript. You can wait for specific selectors, wait for network idle, disable resources for speed, and run custom Playwright page actions like scrolling and clicking.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher makes fast plain HTTP requests for static pages and APIs. DynamicFetcher runs a real browser for JS-rendered content. StealthyFetcher adds anti-fingerprinting and Cloudflare solving for protected sites.

Why does Scrapling fail after pip install?▼

DynamicFetcher and StealthyFetcher fail if browser binaries are missing. After running pip install "scrapling[all]", you must also run scrapling install to download the required browser components.

Does Scrapling support multi-page crawling with resume?▼

Yes, the Spider framework supports concurrent requests, download delays, link following, and multi-session routing. Passing a crawldir directory enables pause and resume, so re-running continues from the last checkpoint.

What Python version does Scrapling require?▼

Scrapling requires Python 3.10 or newer. Install it with pip install "scrapling[all]" for full functionality, or pip install scrapling for a minimal HTTP-only setup without browser automation.

Related Skills

Explore More Software Engineering Skills→
ZJU-REALZJU-REAL
195

firecrawl-map

Discover and filter website URLs using the Firecrawl command-line interface.

Official
Basic
TurnaboutHeroTurnaboutHero
12

data-wizard

Automate ETL and data transformation for CSV, JSON, and database sources using Python and pandas.

Community
Advanced
actionbookactionbook
1.6k

extract

Extract structured website data and generate a standalone Playwright extraction script.

Official
Advanced
mazreanmazrean

building-static-site-rss-feeds

Generate RSS 2.0, Atom 1.0, and JSON feeds for static sites.

Community
Intermediate