skills.rest
Tommy-yw
Tommy-ywCOMMUNITY
Explore all skills by this author →
GitHub

scrapling

Extract web data with stealthy multi-method fetching for Python.

539|39|Updated May 1, 2026
Search Tags:#browser automation#web scraping#stealth#cloudflare bypass#spider crawling
One-click install
npx skills add https://github.com/Tommy-yw/RunbookHermes --skill scrapling-tommy-yw
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/Tommy-yw/RunbookHermes/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/Tommy-yw/RunbookHermes --skill scrapling-tommy-yw

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling[all], requests, beautifulsoup4, lxml, aiohttp, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a comprehensive solution for web scraping, bypassing anti-bot measures, and automating data retrieval from various sources.

Core Features & Use Cases

  • Stealthy Scraping: Overcome Cloudflare and bot detection to access protected content.
  • Multi-Method Fetching: Supports HTTP, dynamic JS, and stealth modes for different page types.
  • Spider Framework: Enables crawling and following links across multiple pages.
  • Use Case: Use Scrapling to collect data from e-commerce sites, social media platforms, or any other public website for research or data analysis.

Quick Start

Use the scrapling skill to extract data from 'https://example.com' and save it to 'output.json'.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I bypass Cloudflare anti-bot protection when web scraping?▼

Scrapling provides stealth fetching modes designed to bypass Cloudflare and overcome anti-bot detection during web scraping. It simulates human behavior to successfully access and extract protected website content.

Can I scrape dynamically loaded JavaScript content with Python?▼

Yes, Scrapling supports dynamic JS fetching strategies to scrape dynamically loaded JavaScript content with Python. This renders asynchronous page elements to ensure complete automated data retrieval.

What is the best way to crawl multiple pages and follow links for data collection?▼

Using a spider crawling framework is the best way to crawl multiple pages and follow links for data collection. Scrapling includes this functionality to automate navigating across page sequences for systematic research.

Does Scrapling require specific Python libraries for web automation?▼

Yes, Scrapling requires Python and specific scraping libraries including requests, beautifulsoup4, lxml, and aiohttp. Installing the scrapling[all] dependency ensures all necessary modules for web automation are configured.

When should I use HTTP fetching versus stealth modes for web scraping?▼

Use HTTP fetching for standard static pages without bot protection, and switch to stealth modes for websites protected by Cloudflare. Scrapling supports multiple fetching strategies to handle different page types efficiently.

Related Skills

Explore More Data & Analytics Skills→
IhsanDanish25IhsanDanish25
2

theme-detector

Detect and analyze trending market themes across sectors with confidence scores.

Community
Advanced
amychina12amychina12

paper-standardizer

Standardize raw academic paper data into analysis-ready datasets with Python codebooks.

Community
Advanced
manikostomanikosto
46

async-http-patterns

Provide reusable async HTTP client patterns for httpx and aiohttp.

Community
Advanced
ldzhhxxldzhhxx
1

scrapling

Automate web scraping with stealth mode and Python libraries.

Community
Advanced