skills.rest
JamesFincher
JamesFincherCOMMUNITY
Explore all skills by this author →
GitHub

scrapling

Fetch static, JavaScript-rendered, and stealth-protected web pages with CSS selectors.

Updated May 4, 2026
Search Tags:#web scraping#javascript rendering#cloudflare bypass#stealth browsing#python automation#spider crawling#cli extraction
One-click install
npx skills add https://github.com/JamesFincher/gengar --skill scrapling-jamesfincher
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/JamesFincher/gengar/tree/main/optional-skills/research/scrapling
Command: npx skills add https://github.com/JamesFincher/gengar --skill scrapling-jamesfincher

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Scrapling eliminates the frustration of getting consistent website data when pages are dynamic, protected by anti-bot systems, or require crawling multiple linked pages.

Core Features & Use Cases

  • Fetch static, dynamic, and stealth-protected pages: Use HTTP for fast static extraction, a browser-backed dynamic fetcher for JavaScript-rendered content, and stealth/Cloudflare bypass strategies when sites block automation.
  • Run targeted extraction with CSS selectors: Retrieve specific elements (text, attributes, lists) and shape outputs based on extensions like .md, .html, .txt, .json, or .jsonl.
  • Spider crawling for multi-page workflows: Follow links across pages, support concurrent crawling, and pause/resume via checkpoints for long-running research.
  • Use Case: Gather product or listing data from a site that renders content client-side and uses anti-bot checks, then crawl through pagination until you have a dataset you can export for analysis.

Quick Start

Use the scrapling skill to extract a protected website into an output file named results.html.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape websites that use Cloudflare or Turnstile anti-bot protection?▼

Scrape Cloudflare or Turnstile protected pages by configuring stealth anti-bot behaviors. Use stealth fetch strategies to bypass blocks and retrieve dynamic content rendered client-side. Select a stealth fetcher to reliably return data from pages blocking standard automation.

How do I crawl multiple pages with spider-style link following?▼

Crawl multi-page sites by running spider-style link following with concurrent crawling. Configure checkpoints to pause and resume long-running research workflows. This approach gathers linked data across pagination into a single exportable dataset.

How do I extract specific elements using CSS selectors and export to JSON?▼

Extract specific elements using CSS selectors to retrieve text, attributes, or lists. Export results to JSON or JSONL by naming the output file with the corresponding extension. The output format is inferred directly from the destination file extension.

Can I scrape JavaScript-rendered content from dynamic websites?▼

Scrape JavaScript-rendered content by using a browser-backed dynamic fetcher. This retrieves client-side rendered data that standard HTTP requests miss. Use this fetch strategy when pages require executing scripts to display target data.

What is the best way to scrape static pages quickly without browser overhead?▼

Scrape static pages quickly by using a standard HTTP fetch strategy. This avoids browser overhead and provides fast extraction for pages without JavaScript rendering or anti-bot protection. Select this fetcher for simple, static content retrieval.

Do I need Python automation knowledge to use this web scraping framework?▼

Python automation knowledge helps but is optional. Use the CLI for direct extraction or the Python framework for complex spider crawling. Configure fetch strategies and stealth behaviors through either interface to produce formatted outputs.

Related Skills

Explore More Education & Research Skills→
raphaelchristiraphaelchristi
43

harness:setup

Configure Harness Evolver, LangSmith, and baseline evaluations for Python projects.

Community
Advanced
Wesley1600Wesley1600

data-analysis

Analyze CSV, JSON, and Parquet datasets with exploratory analysis and visualizations.

Community
Advanced
FutureAtomsFutureAtoms
1

vercel-browser-kernel

Configure Browserbase Kernel cloud browsers for stealth browsing and persistent sessions.

Community
Advanced
vm0-aivm0-ai
76

scrapeninja

Extract data from anti-bot-protected websites via a scraping API.

Official
Intermediate