skills.rest
walker-hzx
walker-hzxCOMMUNITY
Explore all skills by this author →
GitHub

web-researcher

Crawl specified URLs with Playwright and output Markdown or JSON results.

Updated Feb 19, 2026
Search Tags:#automation#playwright#web#data-extraction#structured-data#crawl#information-retrieval
One-click install
npx skills add https://github.com/walker-hzx/AI-Assistant --skill web-researcher-walker-hzx
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-researcher
Source: https://github.com/walker-hzx/AI-Assistant/tree/main/skills/web-researcher
Command: npx skills add https://github.com/walker-hzx/AI-Assistant --skill web-researcher-walker-hzx

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Crawls specified URLs to extract key information and analyze content.

Core Features & Use Cases

  • Single-page crawl: fetch a single page and extract key data.
  • Site-wide crawl: automatically discover and crawl all related pages.
  • Module-specific crawl: target specific site paths for focused research.
  • Batch crawl: process multiple URLs from a list for comparative analysis.
  • Intelligent analysis: structured extraction and interpretation of page content.

Quick Start

Provide a URL to crawl and the agent will fetch, crawl, and extract key insights automatically.

Frequently Asked Questions about web-researcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from dynamic web pages?▼

You can extract structured data from dynamic web pages by using Playwright to render JavaScript, allowing the crawler to fetch dynamic content and output structured Markdown or JSON results.

What is the best way to crawl an entire site for research?▼

The best way to crawl an entire site is using the site-wide crawl feature, which automatically discovers related pages and extracts key insights for comprehensive research workflows.

Can I batch process multiple URLs to extract information?▼

Yes, you can batch process multiple URLs from a list for comparative analysis, fetching key data from each page and outputting structured Markdown or JSON results.

Does this web crawler work with Playwright for JavaScript rendering?▼

Yes, this web crawler works with Playwright for dynamic content handling, ensuring JavaScript-rendered pages are fully loaded before extracting key information and insights.

How do I target specific site paths for focused web extraction?▼

You can target specific site paths for focused web extraction by using module-specific crawl modes, isolating data retrieval to defined site sections and outputting structured results.

What output formats are supported for downstream AI tasks?▼

The supported output formats for downstream AI tasks are Markdown and JSON, ensuring extracted web data and key insights are structured for immediate integration into automated workflows.

Related Skills

Explore More Data & Analytics Skills→
jscraikjscraik
8

cf-crawl

Initiate, monitor, and export website crawls via the Cloudflare Browser Rendering API.

Community
Advanced
pacphipacphi
14

wardley-mapper

Transform inputs into Wardley maps with automatic strategic analysis.

Community
Advanced
timequitytimequity
6

data-quality

Detect data quality issues in dbt and Great Expectations pipelines.

Official
Advanced
zhayujiezhayujie
46.4k

web-fetch

Fetch web page HTML and extract title and readable text.

Community
Basic