skills.rest
Rawgrowth-Consulting
Rawgrowth-ConsultingOFFICIAL
Explore all skills by this author →
GitHub

scrapling

Automate web scraping with anti-bot bypass and dynamic JS rendering.

Updated Apr 23, 2026
Search Tags:#cli#cloudflare#web-scraping#browser-automation#spider#stealth#scrapling
One-click install
npx skills add https://github.com/Rawgrowth-Consulting/rawclaw-agent --skill scrapling-rawgrowth-consulting
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/Rawgrowth-Consulting/rawclaw-agent/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/Rawgrowth-Consulting/rawclaw-agent --skill scrapling-rawgrowth-consulting

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Scrapling provides a robust framework to automate web scraping in environments where bot defenses and JavaScript-heavy pages block simple requests. It offers HTTP fetching, dynamic JS rendering, stealth bypass, and a spider framework to crawl multiple pages with respectful pacing.

Core Features & Use Cases

  • HTTP, dynamic JS, and stealth fetchers to handle a wide range of targets
  • Spider framework for multi-page crawling with link following
  • CLI and Python APIs for easy integration into data pipelines and research workflows
  • Cloudflare bypass capabilities with configurable safety checks and usage guidelines

Quick Start

Install Scrapling and start a basic crawl to fetch data from a target site using the CLI or Python interface.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape Cloudflare-protected websites with anti-bot bypass?▼

You can scrape Cloudflare-protected websites by using stealth fetchers with anti-bot bypass capabilities. Scrapling provides configurable safety checks and usage guidelines specifically for handling Cloudflare defenses during web scraping.

What is the best way to crawl multiple pages following links with dynamic JS rendering?▼

The best way to crawl multiple pages with dynamic JS rendering is using a spider framework. Scrapling automates multi-page crawling with link following and respectful pacing to extract data across JavaScript-heavy sites.

Can I integrate web scraping into Python data pipelines using a CLI?▼

Yes, you can integrate web scraping into Python data pipelines using both CLI tools and Python APIs. Scrapling provides HTTP fetching and stealth bypass capabilities for easy integration into research workflows and data pipelines.

When do I need stealth fetchers versus standard HTTP fetching for web scraping?▼

You need stealth fetchers when scraping JavaScript-heavy sites or targets with bot defenses, while standard HTTP fetching works for static pages. Scrapling offers both options to handle a wide range of web scraping targets.

Does web scraping with spider-based crawling support multi-page workflows?▼

Yes, spider-based crawling supports multi-page workflows by following links across pages. Scrapling automates this process with respectful pacing to extract data efficiently across complex website structures.

What are the limitations of anti-bot bypass for Cloudflare-protected targets?▼

Anti-bot bypass for Cloudflare-protected targets requires adherence to configurable safety checks and documented usage guidelines. Scrapling provides clear safety notes and prerequisites to ensure responsible scraping of protected sites.

Related Skills

Explore More Software Engineering Skills→
babamba2babamba2
46

sc4sap:mcp-setup

Install and configure the abap-mcp-adt-powerup MCP server for SAP connectivity.

Community
Advanced
Pans0020Pans0020

markdown-exporter

Convert Markdown content into DOCX, PPTX, XLSX, PDF, HTML, IPYNB, MD, CSV, JSON/JSONL, XML, and extracted code files.

Community
Advanced
EdwardsagaEdwardsaga
8

agent-stealth-browser

Automate web browser interactions with stealth to bypass anti-bot detection.

Community
Advanced
Chris-Chai-MinjaeChris-Chai-Minjae

scrapling

Automate web scraping with HTTP fetching, stealth browser automation, and spider crawling.

Community
Advanced