web-researcher

Name: web-researcher
Availability: InStock
Author: walker

Crawl specified URLs with Playwright and output Markdown or JSON results.

Updated Feb 19, 2026

Search Tags:#automation #playwright #web #data-extraction #structured-data #crawl #information-retrieval

Ready to run

npx skills add https://github.com/walker-hzx/AI-Assistant --skill web-researcher-walker-hzx

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-researcher
Source: https://github.com/walker-hzx/AI-Assistant/tree/main/skills/web-researcher
Command: npx skills add https://github.com/walker-hzx/AI-Assistant --skill web-researcher-walker-hzx

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Crawls specified URLs to extract key information and analyze content.

Core Features & Use Cases

Single-page crawl: fetch a single page and extract key data.
Site-wide crawl: automatically discover and crawl all related pages.
Module-specific crawl: target specific site paths for focused research.
Batch crawl: process multiple URLs from a list for comparative analysis.
Intelligent analysis: structured extraction and interpretation of page content.

Quick Start

Provide a URL to crawl and the agent will fetch, crawl, and extract key insights automatically.

Frequently Asked Questions about web-researcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract structured data from dynamic web pages?▼

You can extract structured data from dynamic web pages by using Playwright to render JavaScript, allowing the crawler to fetch dynamic content and output structured Markdown or JSON results.

What is the best way to crawl an entire site for research?▼

The best way to crawl an entire site is using the site-wide crawl feature, which automatically discovers related pages and extracts key insights for comprehensive research workflows.

Can I batch process multiple URLs to extract information?▼

Yes, you can batch process multiple URLs from a list for comparative analysis, fetching key data from each page and outputting structured Markdown or JSON results.

Does this web crawler work with Playwright for JavaScript rendering?▼

Yes, this web crawler works with Playwright for dynamic content handling, ensuring JavaScript-rendered pages are fully loaded before extracting key information and insights.

How do I target specific site paths for focused web extraction?▼

You can target specific site paths for focused web extraction by using module-specific crawl modes, isolating data retrieval to defined site sections and outputting structured results.

What output formats are supported for downstream AI tasks?▼

The supported output formats for downstream AI tasks are Markdown and JSON, ensuring extracted web data and key insights are structured for immediate integration into automated workflows.