scrapling

Scrapes web pages using HTTP fetching, stealth browser automation, and spider crawling.

Updated Jun 7, 2026
One-click install
npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill scrapling-chensihakniroth
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/Chensihakniroth/ANAKOT-AGENT/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill scrapling-chensihakniroth

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering, sit behind Cloudflare protection, or span many linked pages. This Skill provides CLI and Python workflows for scraping static, dynamic, and anti-bot-protected sites with the Scrapling framework. ## Core Features & Use Cases - Three Fetching Strategies: Use Fetcher for fast static HTTP requests, DynamicFetcher for JS-rendered SPAs, and StealthyFetcher to bypass Cloudflare Turnstile and bot detection. - Spider Crawling Framework: Build multi-page crawlers with link following, concurrent requests, multi-session routing, and pause/resume checkpoints. - Rich Element Selection: Query pages with CSS selectors, XPath, text/regex finders, and similar-element detection for product listings. - Use Case: Imagine you need to collect quotes from a paginated site protected by Cloudflare. Use the stealth fetch mode to bypass the challenge, then run a Spider to follow pagination links and export results to JSON. ## Quick Start Use the scrapling skill to extract the main content from https://example.com into a Markdown file.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a Cloudflare-protected website in Python?▼

Use Scrapling's StealthyFetcher with solve_cloudflare=True to bypass Cloudflare Turnstile challenges. Options like block_webrtc and hide_canvas reduce browser fingerprinting, though solving adds 5-15 seconds per fetch.

How to scrape JavaScript-rendered pages with Scrapling?▼

Use DynamicFetcher or DynamicSession to load pages in a real headless browser that executes JavaScript. You can wait for specific selectors, enable network_idle, and disable resources like images and fonts for roughly 25% faster loads.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher makes fast plain HTTP requests for static pages and APIs. DynamicFetcher runs a real browser for JS-rendered content. StealthyFetcher adds anti-fingerprinting and Cloudflare bypass for protected sites.

Does Scrapling support multi-page crawling with pagination?▼

Yes, the Spider framework supports concurrent requests, download delays, link following via response.follow, and pause/resume checkpoints using a crawldir. You can also route requests to different session types within one spider.

Why does Scrapling DynamicFetcher fail after pip install?▼

DynamicFetcher and StealthyFetcher fail because browser binaries are missing. Run scrapling install after pip install to download the required browser dependencies before using browser-based fetchers.

What Python version does Scrapling require?▼

Scrapling requires Python 3.10 or newer. Install it with pip install "scrapling[all]" for full functionality, or use the minimal install for HTTP-only fetching without browser support.