smart-web-scraper
CommunityRobust web scraping with anti-bot protection.
Authorhaomingz
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Automates extracting content from dynamic websites that deploy anti-bot measures, making it easy to obtain page titles, text content, and screenshots without manual scraping.
Core Features & Use Cases
- Built-in Playwright-based simple and stealth modes to adapt to websites with JavaScript rendering and Cloudflare protections.
- Outputs structured data including title, URL, content preview, and optional screenshot/HTML.
- Use cases include collecting product pages, news articles, or forum threads where automation is needed with resilience to anti-bot defenses.
Quick Start
Scrape a target webpage using the stealth script and return the title, content, and a screenshot.
Dependency Matrix
Required Modules
playwright
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: smart-web-scraper Download link: https://github.com/haomingz/kimi-skills/archive/main.zip#smart-web-scraper Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.