audit-crawlability

Analyze robots.txt directives, URL status codes, redirect chains, and crawl depth.

Updated Apr 26, 2026
One-click install
npx skills add https://github.com/seohow/seo --skill audit-crawlability
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: audit-crawlability
Source: https://github.com/seohow/seo/tree/main/skills/technical/audit-crawlability
Command: npx skills add https://github.com/seohow/seo --skill audit-crawlability

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps identify and resolve crawlability problems on a website, ensuring search engines can access and index content properly.

Core Features & Use Cases

  • Crawlability Diagnostics: Analyze robots.txt and URL lists to detect blocking patterns, redirect issues, and crawl depth problems.
  • Issue Prioritization: Highlight high-impact problems on key pages to guide effective remediation.
  • Use Case: A website owner uploads a list of URLs and their robots.txt, then receives a detailed report on crawl issues and recommended fixes, ensuring better indexation.

Quick Start

Provide the URLs of your robots.txt file and list of site URLs to generate a comprehensive crawlability audit report.

Frequently Asked Questions about audit-crawlability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose website crawlability issues with robots.txt and URL status codes?

Diagnose website crawlability by analyzing your robots.txt directives and URL status codes to detect blocking patterns, redirect issues, and crawl depth problems. Provide your robots.txt file and a list of site URLs to generate a detailed audit report.

What causes orphan pages and how do I fix them in a site audit?

Orphan pages occur when URLs lack internal links, preventing search engines from discovering them. A site audit identifies these pages by evaluating crawl depth and internal link structures, recommending fixes to ensure better indexation.

How do I identify and resolve redirect chains affecting search engine accessibility?

Identify redirect chains by evaluating URL status codes to spot multiple sequential redirects that waste crawl budget. The audit highlights these issues and recommends direct linking to resolve redirect loops and optimize search engine accessibility.

Can I use a URL list to detect noindex tags and crawl depth problems?

Yes, you can use a URL list to detect noindex tags and crawl depth problems. The analysis evaluates the provided URLs against robots.txt rules to highlight pages blocked from indexing and those buried too deep in the site architecture.

What is the best way to prioritize crawlability issues on key pages?

The best way to prioritize crawlability issues is through an audit that highlights high-impact problems on key pages. It guides effective remediation by focusing on critical blocking patterns and redirect issues that hinder search engine accessibility.

When should I run a crawlability audit to improve indexation?

You should run a crawlability audit when experiencing poor indexation or dropping rankings. It detects blocking patterns in robots.txt, noindex tags, and crawl depth problems to ensure search engines can access and index your content properly.