skills.rest
BermudaLocals
BermudaLocalsCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

scrapling

Automate web scraping with HTTP, dynamic, and stealth fetchers.

1|1|Updated Apr 26, 2026
Search Tags:#python#browser automation#web scraping#spider#stealth#cloudflare bypass
One-click install
npx skills add https://github.com/BermudaLocals/hermes-agent-lite --skill scrapling-bermudalocals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/BermudaLocals/hermes-agent-lite/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/BermudaLocals/hermes-agent-lite --skill scrapling-bermudalocals

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web scraping often confronts bot protections, dynamic content, and complex crawls. Scrapling provides HTTP, dynamic, and stealth fetching strategies plus a Spider framework to simplify data extraction at scale.

Core Features & Use Cases

  • Automated fetching strategies: HTTP, dynamic JS, and stealth fetchers to handle static and protected pages.
  • Spider framework: Multi-page crawling with link following for scalable scraping pipelines.
  • CLI & Python API: Flexible usage from the command line or Python scripts to automate extraction tasks.

Quick Start

Install Scrapling with pip and run a sample fetch to extract data from a target site.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape websites that use Cloudflare bot protection?▼

To bypass Cloudflare bot protection during web scraping, you use a stealth fetcher mode. This strategy handles protected pages by rendering dynamic content while evading detection in production or research contexts.

What is the best way to crawl multiple pages with dynamic JavaScript content?▼

Crawling multiple pages with dynamic JavaScript content requires a spider framework combined with dynamic fetchers. This approach automates link following and JS-rendered data extraction for scalable scraping pipelines.

Can I run web scraping spiders from the command line?▼

Yes, you can run web scraping spiders from the command line. The framework provides a CLI to configure fetchers and execute multi-page crawls, alongside a Python API for script-based automation.

Do I need Python to automate web scraping tasks with this framework?▼

Yes, you need Python to automate web scraping tasks with this framework. It requires the Python environment, the Scrapling package installed via pip, and a CLI to configure fetchers and run spiders.

When should I use HTTP fetching versus stealth fetching for web scraping?▼

Use HTTP fetching for static web scraping pages, and stealth fetching for protected sites. Dynamic fetching handles JS-rendered content, providing a unified framework to match your extraction strategy to the target site.

Related Skills

Explore More Software Engineering Skills→
Chris-Chai-MinjaeChris-Chai-Minjae

scrapling

Automate web scraping with HTTP fetching, stealth browser automation, and spider crawling.

Community
Advanced
Pans0020Pans0020

markdown-exporter

Convert Markdown content into DOCX, PPTX, XLSX, PDF, HTML, IPYNB, MD, CSV, JSON/JSONL, XML, and extracted code files.

Community
Advanced
darblexdarblex

web-scraping

Fetch and extract web page data using Node.js fetch and regex parsing.

Community
Intermediate
iArsalanshahiArsalanshah
7

ghostfetch

Extract webpage content from protected sites using a stealthy headless browser.

Community
Intermediate