site-content-catalog

Crawl sitemaps and blog indexes to generate a categorized content inventory.

1.1k|200|Updated Mar 2, 2026
One-click install
npx skills add https://github.com/athina-ai/goose-skills --skill site-content-catalog
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: site-content-catalog
Source: https://github.com/athina-ai/goose-skills/tree/main/skills/capabilities/site-content-catalog
Command: npx skills add https://github.com/athina-ai/goose-skills --skill site-content-catalog

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests, and includes scripts (resource) components.

What problem does it solve?

This Skill automates the process of discovering and inventorying all content on a website, making it easier to understand your content landscape for SEO, gap analysis, or brand voice extraction.

Core Features & Use Cases

  • Comprehensive Crawling: Discovers pages via sitemaps, RSS feeds, and blog index crawling.
  • Content Classification: Automatically categorizes pages by type (blog, case study, docs, etc.) and topic cluster.
  • Publishing Analysis: Summarizes content volume, frequency, and recency.
  • Use Case: Before conducting an SEO audit, use this Skill to get a complete list of all blog posts, landing pages, and other content types, along with their topics and publish dates, to identify areas for improvement.

Quick Start

Crawl the website example.com to create a content inventory.

Frequently Asked Questions about site-content-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a content inventory from a website sitemap?

Generating a content inventory from a website sitemap involves crawling the sitemap and blog index to extract page URLs, titles, publish dates, and content types. This process groups pages by category and topic cluster to support SEO audits.

What is topic clustering and how does it work for site analysis?

Topic clustering for site analysis categorizes crawled pages into thematic groups based on content type and subject. It automatically identifies topic clusters across blog posts and landing pages to reveal content gaps and structure.

Can I use Python requests to crawl a website for a content audit?

Yes, you can use Python requests to crawl a website for a content audit. The process uses the requests library to discover pages via sitemaps and RSS feeds, enabling comprehensive site analysis and content cataloging.

What's the best way to automate content discovery for an SEO audit?

The best way to automate content discovery for an SEO audit is to crawl sitemaps and RSS feeds to build a complete content inventory. This approach captures URLs, titles, and publish dates while grouping content by topic clusters.

Does deep analysis of top pages identify funnel stages for content strategy?

Yes, deep analysis of top pages identifies funnel stages for content strategy. By performing quality assessment on top pages, the process tags content with funnel stages to support targeted SEO audits and gap analysis.

When do I need a full website crawl instead of manual content inventory?

You need a full website crawl instead of a manual content inventory when managing large sites with diverse content types like blogs and case studies. Automated crawling discovers all pages via sitemaps to ensure complete gap analysis.