What problem does it solve? Web scraping often fails when sites use JavaScript rendering, Cloudflare Turnstile, or other anti-bot protections, and scrapers break when page structures change. This Skill provides CLI commands, Python APIs, and a spider framework to fetch, parse, and crawl websites reliably, including protected and dynamic pages. ## Core Features & Use Cases - Tiered Fetching: Escalate from plain HTTP requests (get) to browser rendering (fetch) to stealth mode with Cloudflare solving (stealthy-fetch) depending on site protection. - Adaptive Parsing: Automatically relocate elements when websites change structure, using saved element fingerprints stored in SQLite. - Spider Framework: Build concurrent multi-session crawlers with pause/resume checkpoints, proxy rotation, robots.txt compliance, and JSON export. - Use Case: Extract all product listings from a Cloudflare-protected e-commerce site by running scrapling extract stealthy-fetch with --solve-cloudflare, or write a Spider class that follows pagination and exports results to JSON. ## Quick Start Install Scrapling with pip, then ask the agent to scrape a target URL using the scrapling extract get command and save the output as a Markdown file.
