link-crawler
CommunityAudit website links, find dead URLs in seconds.
Software Engineering#web scraping#url validation#sitemap generation#website audit#link verification#link crawling#broken link check
Authorvenhdev
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Manually checking if every link on a website works is extremely time-consuming for sites with hundreds or thousands of pages, and getting a full inventory of all active content without automated tools is nearly impossible.
Core Features & Use Cases
- Automated Link Crawling: Crawl websites up to 3 levels deep to discover all linked pages from one or more root URLs.
- Link Validation: Automatically verify each discovered link returns a valid, content-rich page, filtering out error pages, placeholders, and dead links.
- Use Case: Use this to audit a documentation site for broken links before a release, generate a full sitemap of active pages for a product website, or verify all links remain working after a site migration.
Quick Start
Use the link-crawler skill to crawl a specified website to your chosen depth and get a grouped list of all active working links along with counts of any dead or unreachable URLs.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: link-crawler Download link: https://github.com/venhdev/skills/archive/main.zip#link-crawler Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 537,000+ vetted skills library on demand.