Skills
.Work. You
Rest

geo-crawlers

Community

Optimize AI crawler access for visibility.

Marketing & Sales#content strategy#web crawling#seo#visibility#robots.txt#ai crawler
Authorrajains234-netizen
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill ensures your website content is discoverable by AI crawlers, preventing accidental blocking that makes your content invisible in AI-generated responses and search results.

Core Features & Use Cases

  • AI Crawler Audit: Analyzes robots.txt, meta tags, and HTTP headers for major AI crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.).
  • Access Mapping: Provides a clear map of which AI crawlers can access your site and which are blocked.
  • Recommendation Engine: Offers specific, actionable recommendations for configuring robots.txt and other directives to maximize AI visibility.
  • Use Case: A content publisher uses this Skill to identify that their robots.txt is inadvertently blocking GPTBot, leading to their content not appearing in ChatGPT's AI search results. The Skill provides the exact Allow: / directive needed to fix this.

Quick Start

Analyze the AI crawler access for the domain example.com.

Dependency Matrix

Required Modules

None required

Components

scriptsreferences

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: geo-crawlers
Download link: https://github.com/rajains234-netizen/GEO/archive/main.zip#geo-crawlers

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.