skills.rest
ossoolli
ossoolliCOMMUNITY
Explore all skills by this author →
GitHub

scrapling

Scrapes web pages using HTTP fetching, stealth browser automation, and spider crawling.

Updated May 18, 2026
Search Tags:#data extraction#python#web scraping#scrapling#cloudflare bypass#stealth browser#spider crawling
One-click install
npx skills add https://github.com/ossoolli/Nexum-Core --skill scrapling-ossoolli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/ossoolli/Nexum-Core/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/ossoolli/Nexum-Core --skill scrapling-ossoolli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering or are protected by anti-bot systems like Cloudflare, and built-in extraction tools return incomplete results. ## Core Features & Use Cases - Three Fetching Strategies: Use Fetcher for static pages, DynamicFetcher for JS-rendered SPAs, and StealthyFetcher for Cloudflare-protected sites. - Spider Framework: Crawl multiple pages with link following, concurrent requests, multi-session routing, and pause/resume checkpoints. - CLI and Python APIs: Extract pages directly from the command line into HTML, Markdown, text, or JSON, or script complex scraping in Python. - Use Case: Scrape a paginated quotes site by writing a Spider class that yields structured items, follows next-page links, and exports results to JSON. ## Quick Start Use the scrapling skill to extract the main content from https://example.com into a Markdown file.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a Cloudflare-protected website in Python?▼

Use Scrapling's StealthyFetcher with solve_cloudflare=True to bypass Cloudflare Turnstile and bot detection. Options like block_webrtc and hide_canvas reduce browser fingerprinting, though solving adds 5-15 seconds per fetch.

How to scrape JavaScript-rendered pages without Selenium?▼

Use Scrapling's DynamicFetcher, which runs a real browser to execute JavaScript and render SPAs. You can wait for specific selectors, block resources for speed, and inject custom Playwright page actions like scrolling and clicking.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher makes fast HTTP requests for static pages, DynamicFetcher renders JS-heavy pages in a browser, and StealthyFetcher adds anti-fingerprinting and Cloudflare solving. Choose based on the target site's protection level.

Why does Scrapling DynamicFetcher fail after pip install?▼

DynamicFetcher and StealthyFetcher fail because browser binaries are missing. Run 'scrapling install' after pip installation to download the required browser dependencies before using browser-based fetchers.

Can Scrapling spiders pause and resume crawling?▼

Yes, pass a crawldir parameter to the Spider constructor to enable checkpointing. Interrupt the crawl with Ctrl+C and re-run the same script to resume from the saved state.

What are the limitations of web scraping with Scrapling?▼

StealthyFetcher runs a real browser, so concurrent usage consumes significant resources, and Cloudflare solving slows fetches. Users must also respect robots.txt, website Terms of Service, and local scraping laws.

Related Skills

Explore More Software Engineering Skills→
ovisanovisan
2

firecrawl-map

Discovers and lists all URLs on a website with optional search filtering.

Community
Basic
xberg-ioxberg-io
26

extracting-tables

Extract HTML tables and convert them to Markdown and structured JSON.

Official
Intermediate
luis2404123luis2404123

proxy-rotation

Rotate IP addresses per request or sticky session for web scraping.

Community
Intermediate
mazreanmazrean

building-static-site-rss-feeds

Generate RSS 2.0, Atom 1.0, and JSON feeds for static sites.

Community
Intermediate