skills.rest
alinaqi
alinaqiCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

site-architecture

Optimize website architecture for SEO and AI crawler discoverability.

705|56|Updated Dec 26, 2025
Search Tags:#sitemap#seo#core web vitals#meta tags#robots.txt#technical seo#ai crawlers
One-click install
npx skills add https://github.com/alinaqi/claude-bootstrap --skill site-architecture-alinaqi
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: site-architecture
Source: https://github.com/alinaqi/claude-bootstrap/tree/main/skills/site-architecture
Command: npx skills add https://github.com/alinaqi/claude-bootstrap --skill site-architecture-alinaqi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill ensures your website is easily discoverable and understandable by both traditional search engines and emerging AI crawlers, maximizing your content's reach and impact.

Core Features & Use Cases

  • Robots.txt Optimization: Configure rules for search engines and AI bots.
  • Sitemap Generation: Create XML sitemaps for efficient crawling.
  • Meta Tag Implementation: Add essential SEO and social sharing meta tags.
  • URL Structure Best Practices: Maintain clean, SEO-friendly URLs.
  • Core Web Vitals: Guidance on improving site performance metrics.
  • AI Crawler Handling: Specific instructions for bots like GPTBot and ClaudeBot.
  • Use Case: You've launched a new blog and want to ensure Google, ChatGPT, and Claude can all find and index your articles effectively. This skill provides the exact configurations for robots.txt, sitemaps, and meta tags to achieve this.

Quick Start

Configure the robots.txt file to allow all search engines and AI bots while disallowing specific API paths.

Frequently Asked Questions about site-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize robots.txt for AI crawlers like GPTBot and ClaudeBot?▼

Optimizing robots.txt for AI crawlers involves adding specific user-agent rules for bots like GPTBot, ClaudeBot, and PerplexityBot. This ensures your content is accessible to AI search engines while keeping private paths restricted.

What is the best way to structure URLs for technical SEO?▼

The best way to structure URLs for technical SEO is to maintain clean, descriptive, and SEO-friendly paths. This helps search engines understand your content hierarchy and improves overall site architecture discoverability.

How do I generate an XML sitemap for search engine crawling?▼

Generating an XML sitemap involves listing your website's accessible URLs to ensure efficient crawling. This file guides search engines like Google and Bing to index your content effectively.

Does Core Web Vitals impact SEO and how do I improve site performance metrics?▼

Core Web Vitals directly impacts SEO by measuring site performance metrics like loading speed and visual stability. Improving these metrics ensures better search rankings and a smoother user experience.

What meta tags are essential for SEO and social sharing?▼

Essential SEO and social sharing meta tags include Open Graph tags and descriptive title tags. Implementing these correctly ensures search engines and social platforms display your content accurately.

Can I block specific API paths while allowing search engines to crawl my site?▼

Yes, you can block specific API paths by using disallow rules in your robots.txt file. This prevents search engines and AI bots from accessing sensitive endpoints while allowing content crawling.

Related Skills

Explore More Marketing & Sales Skills→
sadie100sadie100

geo-technical

Audit website technical SEO for GEO-specific criteria using Python libraries.

Community
Advanced
AureliusIvanAureliusIvan
2

index-on-search-engine

Register websites and subdomains with Google Search Console and Bing Webmaster Tools.

Community
Intermediate
amplifthqamplifthq
19

page-seo

Audit webpages across 10 on-page SEO factors and generate prioritized fix lists.

Official
Intermediate
TheMattBermanTheMattBerman
92

seo-health

Audit website technical SEO health with weekly automated checks.

Community
Intermediate