scrapling

Automate web scraping with HTTP fetchers, dynamic rendering, and anti-bot bypass.

Updated Apr 19, 2026

Search Tags:#python #cli #cloudflare #web-scraping #dynamic-content #headless-browser #spider-framework

Ready to run

npx skills add https://github.com/gqf2008/hermez-ai --skill scrapling-gqf2008

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/gqf2008/hermez-ai/tree/main/skills/research/scrapling
Command: npx skills add https://github.com/gqf2008/hermez-ai --skill scrapling-gqf2008

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Scrapling reduces the time and effort required for web data collection by combining HTTP requests, dynamic JS rendering, anti-bot bypass, and spider-like crawling into a single, cohesive toolkit that can be used from the CLI or Python.

Core Features & Use Cases

HTTP Fetcher for static pages and APIs
DynamicFetcher for JavaScript-rendered content
StealthyFetcher for Cloudflare-protected sites
Spider framework for multi-page crawling via Python
CLI and Python APIs for flexible integration
Optional browser components for dynamic content

Quick Start

Install Scrapling and run a basic fetch from a site using the CLI or Python API.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape dynamic JavaScript-rendered web pages using Python?▼

To scrape dynamic web pages with Python, you need a headless browser to render JavaScript content. Scrapling provides a DynamicFetcher component that automates this rendering process, returning fully loaded HTML for dynamic sites.

Can I bypass Cloudflare anti-bot protection when web scraping?▼

Yes, you can bypass Cloudflare anti-bot protection using specialized fetching tools. Scrapling includes a StealthyFetcher component designed specifically to circumvent Cloudflare blocks and retrieve protected page content.

What is the best way to crawl multiple pages across a website?▼

The best way to crawl multiple pages is using a spider framework that manages link discovery and traversal. Scrapling offers a built-in spider framework accessible via Python APIs to automate multi-page crawling workflows.

How do I fetch static HTML or retrieve API data from the command line?▼

You can fetch static HTML or API data directly from the command line using an HTTP fetcher tool. Scrapling provides a CLI interface and an HTTP Fetcher component for fast, direct retrieval of static page content.

Do I need a headless browser to scrape static web pages?▼

No, you do not need a headless browser for static web pages or APIs. Scrapling uses a lightweight HTTP Fetcher for fast static content retrieval, while optional browser components are reserved for dynamic rendering.