site-mapping

Extract and classify website URLs from XML sitemaps into a structured content inventory.

115|9|Updated Aug 5, 2026
One-click install
npx skills add https://github.com/AtlasOmnia/donna-starter --skill site-mapping-atlasomnia
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: site-mapping
Source: https://github.com/AtlasOmnia/donna-starter/tree/main/skills/research/site-mapping
Command: npx skills add https://github.com/AtlasOmnia/donna-starter --skill site-mapping-atlasomnia

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Understanding a website's full structure — its sections, URL taxonomy, and content volume — is tedious when done by clicking through pages manually, and rendered sitemap HTML tables truncate after roughly 100 rows, hiding the real picture. ## Core Features & Use Cases - Sitemap Discovery and Parsing: Locates sitemaps via sitemap.xml, robots.txt directives, and common Yoast patterns, then fetches raw XML with browser_console and DOMParser to avoid truncated HTML tables. - URL Classification: Groups URLs by path segments to identify content sections, filtering out media uploads like /wp-content/uploads/ so page counts stay accurate. - Navigation Extraction: Pulls primary nav menus and footer links from the homepage to complement sitemap data. - Use Case: Before a competitive analysis or site migration, map a competitor's site to get a table of sections, URL prefixes, and page counts revealing their content strategy. ## Quick Start Map out the full structure of https://example.com including its sitemap sections, navigation, and page counts per section.

Frequently Asked Questions about site-mapping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract all URLs from a website sitemap?

Fetch the raw sitemap XML using browser_console with fetch() and parse it with DOMParser, reading all loc elements. Check sitemap.xml first, then robots.txt for a Sitemap directive, and look for numbered variants like page-sitemap2.xml.

How to find a website's sitemap if sitemap.xml returns nothing?

Check the site's robots.txt file for a Sitemap directive, which often points to the real location. Also try common patterns like sitemap_index.xml, page-sitemap.xml, and post-sitemap.xml used by Yoast and similar generators.

Why does the sitemap page only show 100 URLs?

Yoast-generated sitemaps render as HTML tables that truncate after roughly 100 rows in the browser. Always fetch the raw XML programmatically with fetch() and DOMParser instead of reading the rendered table.

Can I map Reddit's structure using sitemap extraction?

No, Reddit's robots.txt disallows crawling and its useful structure lives in operational surfaces like old.reddit HTML and .json endpoints, not public XML sitemaps. Use a Reddit-specific browsing approach instead of sitemap crawling.

Why are my sitemap page counts inflated?

Image and media URLs from paths like /wp-content/uploads/ get included in sitemap extractions and pollute counts. Filter out asset upload paths when counting actual content pages.