skills.rest
haomingz
haomingzCOMMUNITY
Explore all skills by this author →
GitHub

smart-web-scraper

Extract titles, content, and screenshots from JavaScript-rendered websites using Playwright.

12|2|Updated Apr 21, 2026
Search Tags:#automation#playwright#cloudflare#web-scraping#anti-detection#dynamic-sites
One-click install
npx skills add https://github.com/haomingz/kimi-skills --skill smart-web-scraper
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: smart-web-scraper
Source: https://github.com/haomingz/kimi-skills/tree/main/skills/smart-web-scraper
Command: npx skills add https://github.com/haomingz/kimi-skills --skill smart-web-scraper

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires playwright, and includes scripts (resource) components.

What problem does it solve?

Automates extracting content from dynamic websites that deploy anti-bot measures, making it easy to obtain page titles, text content, and screenshots without manual scraping.

Core Features & Use Cases

  • Built-in Playwright-based simple and stealth modes to adapt to websites with JavaScript rendering and Cloudflare protections.
  • Outputs structured data including title, URL, content preview, and optional screenshot/HTML.
  • Use cases include collecting product pages, news articles, or forum threads where automation is needed with resilience to anti-bot defenses.

Quick Start

Scrape a target webpage using the stealth script and return the title, content, and a screenshot.

Frequently Asked Questions about smart-web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape dynamic websites that use Cloudflare anti-bot protection?▼

You can scrape dynamic websites protected by Cloudflare using a Playwright-based stealth mode, which handles JavaScript rendering and bypasses basic anti-bot defenses to extract titles, text content, and screenshots.

What is the best way to extract text content from JavaScript-rendered pages?▼

The best way to extract text from JavaScript-rendered pages is using Playwright automation to fully render the dynamic content, allowing you to capture the title, URL, and structured text preview in headless or headed modes.

Does Playwright support capturing screenshots during web scraping automation?▼

Yes, Playwright supports capturing screenshots during web scraping automation, allowing you to output visual page states alongside structured data like titles, URLs, and content previews when extracting target webpages.

Can I customize the user-agent when scraping pages with anti-detection measures?▼

Yes, you can customize the user-agent when scraping pages with anti-detection measures, utilizing simple and stealth modes alongside headless or headed configurations to adapt to websites deploying JavaScript rendering and Cloudflare defenses.

Why does my web scraper fail on forum threads with JavaScript rendering?▼

Your web scraper fails on forum threads with JavaScript rendering because basic scrapers cannot execute JavaScript or bypass Cloudflare defenses, requiring a Playwright-based stealth mode to properly render dynamic content and extract data.

Related Skills

Explore More Software Engineering Skills→
aresbitaresbit
46

agent-browser

Automate browser navigation, form filling, and data extraction via CLI.

Community
Intermediate
profmikegreeneprofmikegreene
1

plan-ingestion

Parses pilot plans and hypotheses into structured JSON or tabular data for downstream analysis.

Community
Intermediate
gaos6egaos6e

web-scraper-as-a-service

Generate, execute, clean, and package web scrapers from briefs.

Community
Advanced
Z43LZ43L

scrapling

Extract web content from static, dynamic, and Cloudflare-protected pages.

Community
Advanced