Skills
.Work. You
Rest

site-investigator

Community

Pre-scraping site explorer for safe automation.

Data & Analytics#automation#reports#playwright#web-scraping#website-analysis#site-investigation
AuthorYH-05
Version1.0.0
Installs0

System Documentation

What problem does it solve?

New developers and data teams often struggle to quickly understand the structure of unknown websites before implementing scraping pipelines. The Site Investigator automates a pre-scraping survey using Playwright MCP to map page structure, selectors, pagination, RSS or sitemap availability, and dynamic behavior, and it outputs structured results in JSON and human-friendly Markdown.

Core Features & Use Cases

  • Systematic site probing to identify page types, selectors, and navigation patterns for reliable scraping.
  • RSS/Sitemap detection, robots.txt analysis, and basic accessibility cues to inform crawl policy.
  • Quick handoff to downstream scrapers with a ready-to-use report for decision-making and implementation.

Quick Start

Inspect a target URL and generate a JSON + Markdown site investigation report.

Dependency Matrix

Required Modules

None required

Components

scriptsreferences

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: site-investigator
Download link: https://github.com/YH-05/quants/archive/main.zip#site-investigator

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.