openclaw-ultra-scraping

Scrape web data with anti-bot bypass and dynamic content rendering.

22|7|Updated Mar 5, 2026

Search Tags:#automation #web scraping #dynamic content #crawling #anti-bot #headless browser

Ready to run

npx skills add https://github.com/LeoYeAI/openclaw-ultra-scraping --skill openclaw-ultra-scraping

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: openclaw-ultra-scraping
Source: https://github.com/LeoYeAI/openclaw-ultra-scraping/tree/main
Command: npx skills add https://github.com/LeoYeAI/openclaw-ultra-scraping --skill openclaw-ultra-scraping

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapy, playwright, beautifulsoup4, api-python-client, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill enables AI assistants to perform robust web scraping and crawling by overcoming anti-bot measures and handling site redesigns seamlessly.

Core Features & Use Cases

Anti-Bot Bypass: Effortlessly bypass Cloudflare, Turnstile, CAPTCHAs, and other anti-detection mechanisms.
Adaptive Element Tracking: Maintain selector accuracy despite webpage reorganization or redesigns.
Large-Scale Crawling: Conduct multi-page crawling with concurrency, pause/resume, and proxy rotation.
Precise Data Extraction: Use CSS and XPath selectors, with support for JavaScript-rendered pages via headless browsers.
Use Case: Collect product data from protected e-commerce sites or aggregate content from dynamic SPA websites automatically.

Quick Start

Ask your AI assistant to scrape product details from a protected site using the stealth mode with adaptive element tracking.

Frequently Asked Questions about openclaw-ultra-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape dynamic content from JavaScript-rendered SPA websites?▼

You can scrape dynamic content from JavaScript-rendered pages using headless browsers. This Skill leverages Playwright to execute JavaScript, render single-page application content, and extract precise data using CSS and XPath selectors.

What's the best way to bypass Cloudflare and CAPTCHAs when web scraping?▼

To bypass anti-bot systems like Cloudflare, Turnstile, and CAPTCHAs during web scraping, this Skill uses stealth mode with custom fetchers. It circumvents anti-detection mechanisms automatically to ensure uninterrupted data collection.

Can I use scrapy for large-scale crawling with proxy rotation and pause/resume capabilities?▼

Yes, you can conduct large-scale crawling with scrapy. This Skill supports multi-page concurrency, proxy rotation, and pause/resume capabilities to handle scalable data extraction workflows efficiently.

How do I maintain web scraping accuracy when a target website redesigns its layout?▼

To maintain scraping accuracy when target websites redesign their layout, this Skill uses adaptive element tracking. It automatically adjusts to webpage reorganizations to keep your CSS and XPath selectors functioning correctly.

Does web scraping with playwright require anti-bot bypass for protected e-commerce sites?▼

Yes, scraping protected e-commerce sites with Playwright often requires anti-bot bypass. This Skill integrates stealth modes with headless browser rendering to evade detection while collecting product data.

Why does web scraping fail on sites with dynamic content and anti-bot protection?▼

Web scraping fails on dynamic sites with anti-bot protection because standard requests cannot execute JavaScript or pass security checks like Cloudflare. This Skill overcomes these barriers using headless browsers and stealth fetchers.