skills.rest
xu1713
xu1713COMMUNITY
Explore all skills by this author →
GitHub

scrapling

Scrapes web pages using HTTP fetching, stealth browser automation, and spider crawling.

Updated Jun 5, 2026
Search Tags:#data extraction#browser automation#web scraping#stealth#scrapling#cloudflare bypass#spider crawling
One-click install
npx skills add https://github.com/xu1713/openhorse --skill scrapling-xu1713
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/xu1713/openhorse/tree/main/openhorse/openhorse/optional-skills/research/scrapling
Command: npx skills add https://github.com/xu1713/openhorse --skill scrapling-xu1713

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering, sit behind Cloudflare or anti-bot protection, or span many linked pages. This Skill provides CLI and Python workflows for fetching static pages, rendering JS-heavy sites, bypassing bot detection, and crawling multi-page targets. ## Core Features & Use Cases - Three Fetching Strategies: Use Fetcher for fast static HTTP requests, DynamicFetcher for JS-rendered SPAs, and StealthyFetcher for Cloudflare-protected sites with Turnstile solving. - Spider Framework: Crawl multiple pages with concurrent requests, link following, multi-session routing, and pause/resume checkpoints. - Rich Element Selection: Query results with CSS selectors, XPath, text/regex search, and similar-element detection for product listings. - Use Case: Scrape a Cloudflare-protected e-commerce catalog by running a stealth fetch with solve_cloudflare enabled, extract product names and prices with CSS selectors, and export results to JSON. ## Quick Start Use the scrapling skill to extract the main content of https://example.com into a Markdown file.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a JavaScript-rendered page in Python?▼

Use Scrapling's DynamicFetcher to fetch JS-rendered pages with a real headless browser. Call DynamicFetcher.fetch with options like wait_selector and network_idle to ensure dynamic content loads before extracting data with CSS selectors.

How to bypass Cloudflare protection when web scraping?▼

Use Scrapling's StealthyFetcher with solve_cloudflare=True to bypass Cloudflare Turnstile and bot detection. Options like block_webrtc and hide_canvas reduce browser fingerprinting, though solving adds 5-15 seconds per fetch.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher makes fast plain HTTP requests for static pages, DynamicFetcher runs a real browser for JavaScript-rendered content, and StealthyFetcher adds anti-fingerprinting and Cloudflare solving for protected sites. Choose based on the target site's defenses.

Why does Scrapling fail after pip install?▼

DynamicFetcher and StealthyFetcher fail if browser binaries are missing. After running pip install scrapling, you must also run the scrapling install command to download the required browser components.

Can Scrapling crawl multiple pages and resume after interruption?▼

Yes, the Spider framework supports concurrent multi-page crawling with link following and pause/resume. Pass a crawldir path to the spider to checkpoint progress, then re-run to resume from where it stopped.

Related Skills

Explore More Software Engineering Skills→
ovisanovisan
2

firecrawl-map

Discovers and lists all URLs on a website with optional search filtering.

Community
Basic
hoanghd218hoanghd218
3

github-trend-finder

Scrape GitHub Trending pages and export repository summaries as JSON and Markdown.

Community
Intermediate
dallascrilleydallascrilley

using-crawl4ai-cli

Extract structured data from websites using the crawl4ai CLI with CSS schemas and LLM extraction.

Community
Advanced
mazreanmazrean

building-static-site-rss-feeds

Generate RSS 2.0, Atom 1.0, and JSON feeds for static sites.

Community
Intermediate