skills.rest
hammercui
hammercuiCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-do

Extract structured JSON data from dynamic web pages via Chrome DevTools Protocol.

Updated Aug 23, 2026
Search Tags:#cdp#data-extraction#web-scraping#browser-automation#json-output#openclaw#javascript-rendering
One-click install
npx skills add https://github.com/hammercui/openclaw-install-cn --skill web-do
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-do
Source: https://github.com/hammercui/openclaw-install-cn/tree/main/04-%E9%AB%98%E7%BA%A7%E9%85%8D%E7%BD%AE/web-do
Command: npx skills add https://github.com/hammercui/openclaw-install-cn --skill web-do

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

CDP网页数据提取需要稳定的浏览器会话和可重复的数据抓取流程，避免直接HTTP请求导致的反爬和数据不一致。

Core Features & Use Cases

  • 通过 CDP 连接到已开启远程调试端口的浏览器，提取页面 DOM/HTML，输出纯 JSON，供 Agent 分析。
  • 支持 JavaScript 渲染页面，确保动态内容可获取；可用于趋势、监控和数据收集场景。
  • 使用场景包括在已登录会话中抓取商品、文章或新闻等页面的结构化数据，并将结果传递给后续流程。

Quick Start

Open a browser with CDP on port 9222 and run the web-do extraction script against a target page to obtain JSON.

Frequently Asked Questions about web-do

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from JavaScript-rendered web pages that block HTTP scraping?▼

You can extract structured data from JavaScript-rendered pages by driving a real browser via Chrome DevTools Protocol (CDP) to access dynamic content and output clean JSON, bypassing HTTP scraping failures.

What is CDP-based web scraping and when should I use it?▼

CDP-based web scraping extracts DOM or HTML data by connecting to a real browser session via Chrome DevTools Protocol, ideal for dynamic pages like dashboards or product listings where direct HTTP requests fail.

How do I scrape a web page using Chrome DevTools Protocol and get JSON output?▼

To scrape a web page and get JSON output, open a CDP-enabled browser on port 9222, then run the extraction script against your target URL to retrieve structured page content as pure JSON.

Do I need a running browser to use CDP for web data extraction?▼

Yes, CDP web data extraction requires a running CDP-enabled browser listening on port 9222 to connect, drive the session, and extract dynamic page content without direct HTTP requests.

Why does direct HTTP scraping fail on dynamic content and how does browser automation help?▼

Direct HTTP scraping fails on dynamic content because it cannot execute JavaScript, whereas browser automation via CDP drives a real browser to render pages fully before extracting structured JSON data.

Can I use CDP web scraping in an already logged-in browser session?▼

Yes, CDP web scraping connects to your existing browser session on port 9222, allowing you to extract structured JSON data from authenticated pages like logged-in dashboards or product listings without re-authenticating.

Related Skills

Explore More Software Engineering Skills→
YeomanYeYeomanYe

cdp-browser-control

Connect to Chrome via CDP to automate browsers while preserving login state.

Community
Intermediate
lanejlanej
40

jq

Parse, filter, and transform JSON data with jq pipelines.

Community
Basic
issdandavisissdandavis
6

hydra-clawbot-synthesis

Coordinate Hydra terminal browsing and multi-agent orchestration for Clawbot/OpenClaw operations.

Community
Advanced
SpaceZephyrSpaceZephyr
333

web-scraper

Fetch URLs and convert HTML page content into readable markdown.

Community
Intermediate