scrapling

Automate web data collection with Python-based fetching, parsing, and crawling.

6|1|Updated Apr 3, 2026

Search Tags:#automation #playwright #adaptive #proxy #web-scraping #spider #scrapling

Ready to run

npx skills add https://github.com/zhouziyue233/great-econometrics --skill scrapling-zhouziyue233

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/zhouziyue233/great-econometrics/tree/main/skills/scrapling
Command: npx skills add https://github.com/zhouziyue233/great-econometrics --skill scrapling-zhouziyue233

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Scrapling unifies fetching, parsing, and crawling to automate web data collection, handling dynamic content and anti-bot protections across sites.

Core Features & Use Cases

Adaptive scraping that preserves selectors across site changes.
Multi-session spiders for large crawls, with proxies and pause/resume.
Anti-bot and JS rendering options via Dynamic/Stealthy fetchers for protected pages.

Quick Start

Install Scrapling and run a minimal crawl to extract sample product data from a test site.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape JavaScript-rendered websites that use anti-bot protection?▼

To scrape JavaScript-rendered websites with anti-bot protection, you can use framework fetchers designed for stealthy browser rendering and multi-session crawling. This approach handles dynamic content while bypassing anti-bot defenses to extract data reliably. It consolidates fetching, parsing, and crawling into a single Python-based framework to automate web data collection across protected pages.

What is the best way to maintain web scraping selectors when a site layout changes?▼

The best way to maintain web scraping selectors when a site layout changes is using an adaptive matching mechanism. This approach automatically preserves element selectors across site updates, ensuring continuous data extraction without manual code adjustments. It handles dynamic content and anti-bot protections across sites to automate fetching, parsing, and crawling.

How do I run a large scale spider crawl with proxy rotation and pause capabilities?▼

To run a large scale spider crawl with proxy rotation and pause capabilities, configure a multi-session spider architecture. This setup supports proxy rotation, checkpointing, and pause/resume functionality to enable scalable, robust scrapes across complex websites. It consolidates data extraction, parsing, and crawling into a unified framework to automate web data collection.

Can I configure per-request selectors and parser settings for a targeted web scraping task?▼

Yes, you can configure per-request selectors and parser settings for targeted web scraping tasks. The framework allows configurability via per-request selectors and parser settings, satisfying requirements for adaptive element matching. It handles static and JavaScript-rendered sites, anti-bot protections, and multi-session crawling to automate web data collection.

Does web scraping with Playwright work for bypassing anti-bot stealth protections?▼

Yes, web scraping with Playwright works for bypassing anti-bot stealth protections when using dynamic and stealthy fetchers. This framework handles JavaScript rendering and anti-bot protections via specialized fetchers for protected pages, enabling scalable, robust scrapes across complex websites while preserving adaptive element matching.