skills.rest
MLT-OSS
MLT-OSSOFFICIAL
Explore all skills by this author →
WebsiteGitHub

scrapling

Scrapes web pages via HTTP, stealth browser automation, and spider crawling with Scrapling.

14|5|Updated Apr 9, 2026
Search Tags:#data extraction#python#browser automation#web scraping#scrapling#cloudflare bypass#spider crawling
One-click install
npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill scrapling-mlt-oss
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/MLT-OSS/hermes-agent-go/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill scrapling-mlt-oss

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering, sit behind Cloudflare or anti-bot protection, or span many linked pages. This Skill provides CLI and Python workflows for HTTP fetching, stealth browser automation, and multi-page crawling so you can reliably collect web data. ## Core Features & Use Cases - Three Fetching Strategies: Use Fetcher for static pages, DynamicFetcher for JS-rendered SPAs, and StealthyFetcher for Cloudflare-protected sites. - Spider Framework: Crawl multiple pages with link following, concurrent requests, multi-session routing, and pause/resume checkpoints. - CLI Extraction: Run one-line commands like scrapling extract stealthy-fetch to save pages as HTML, Markdown, text, or JSON. - Use Case: Scrape a paginated quotes site by defining a Spider class that extracts text, authors, and tags from each page, follows the next-page link, and exports results to JSON. ## Quick Start Use the scrapling skill to extract the main content of a given URL into a Markdown file, choosing stealth mode if the site is Cloudflare-protected.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a Cloudflare-protected website in Python?▼

Use Scrapling's StealthyFetcher with solve_cloudflare=True to bypass Cloudflare Turnstile and bot detection. Options like block_webrtc and hide_canvas reduce browser fingerprinting, though solving adds 5-15 seconds per fetch.

How to scrape JavaScript-rendered pages with Python?▼

Use Scrapling's DynamicFetcher, which runs a real browser to execute JavaScript before extraction. You can wait for specific selectors, enable network_idle, and disable resources like images and fonts for roughly 25% faster loads.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher makes plain HTTP requests for static pages and APIs. DynamicFetcher renders JS-heavy pages in a browser. StealthyFetcher adds anti-fingerprinting and Cloudflare solving for protected sites.

Does Scrapling support multi-page crawling with link following?▼

Yes, the Spider framework supports concurrent requests, download delays, and yielding follow requests to crawl linked pages. It also supports pause/resume via a crawl checkpoint directory and routing requests to different session types.

Why does Scrapling DynamicFetcher fail after pip install?▼

DynamicFetcher and StealthyFetcher require browser binaries installed via the `scrapling install` command after pip installation. Without this step, browser-based fetchers will fail at runtime.

What are the limitations of web scraping with Scrapling?▼

StealthyFetcher runs a real browser, so concurrency must be limited, and Cloudflare solving adds latency. Users must respect robots.txt and website Terms of Service, and Python 3.10 or higher is required.

Related Skills

Explore More Software Engineering Skills→
ovisanovisan
2

firecrawl-map

Discovers and lists all URLs on a website with optional search filtering.

Community
Basic
TurnaboutHeroTurnaboutHero
12

data-wizard

Automate ETL and data transformation for CSV, JSON, and database sources using Python and pandas.

Community
Advanced
iAziz786iAziz786
3

firecrawl

Searches, scrapes, and interacts with web pages via a local Firecrawl API.

Community
Advanced
timurgaleevtimurgaleev
5

scrape

Extract structured data from web pages into JSON using a read-only browse shim.

Community
Intermediate