skills.rest
chapter42
chapter42COMMUNITY
Explore all skills by this author →
WebsiteGitHub

42-screaming-frog

Automate headless Screaming Frog CLI crawls and export crawl data.

Updated Aug 23, 2026
Search Tags:#cli#audit#seo#redirects#crawl#indexability#screamingfrog
One-click install
npx skills add https://github.com/chapter42/SEO-Skills-42 --skill 42-screaming-frog
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: 42-screaming-frog
Source: https://github.com/chapter42/SEO-Skills-42/tree/main/skills/42-screaming-frog
Command: npx skills add https://github.com/chapter42/SEO-Skills-42 --skill 42-screaming-frog

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Running Screaming Frog crawls traditionally relies on GUI or brittle workarounds; this skill enables CLI-driven, headless crawls with automated exports, fitting into Claude Code workflows and 42: skill orchestration.

Core Features & Use Cases

  • CLI-based crawl control with commands for crawl, status, export, and analysis.
  • Profile-driven crawls using standard Screaming Frog .seospider profiles to tailor technical audits, content checks, and link analysis.
  • Rich data exports across multiple tabs (Internal, Canonicals, Hreflang, Images, Sitemaps, etc.) and optional Python API for programmatic insights.
  • Integrates with MCP server and Claude Code workflows to feed crawl data into other 42: skills.

Quick Start

Start a headless crawl with a chosen profile and export the results to a local folder for downstream analysis.

Frequently Asked Questions about 42-screaming-frog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run headless Screaming Frog crawls from the command line?▼

This Skill automates headless Screaming Frog crawls via the command line, enabling you to initiate site audits, check crawl status, and export technical SEO data without GUI interaction.

Can I use Screaming Frog CLI with custom .seospider profile files?▼

Yes, crawls are profile-driven using standard .seospider files, allowing you to apply tailored configurations for technical audits, content checks, and link analysis directly in headless mode.

What SEO data exports are available when crawling sites with Screaming Frog CLI?▼

CLI-driven exports cover multiple data tabs including Internal, Canonicals, Hreflang, Images, and Sitemaps, with an optional Python API available for deeper programmatic insights.

Does Screaming Frog CLI automation work with MCP server and Claude Code workflows?▼

Yes, the automation integrates with MCP server and Claude Code workflows to feed exported crawl data directly into downstream skills for cross-project comparison and extended analysis.

What do I need to automate scalable SEO audits with Screaming Frog CLI?▼

You need the Screaming Frog CLI installed locally along with configured .seospider profile files; an optional Python API or MCP setup enables extended programmatic analysis and orchestration.

Why use command line Screaming Frog automation instead of the GUI for SEO audits?▼

CLI automation enables headless, scalable site audits and cross-project comparisons within automated workflows, avoiding the manual overhead and brittle workarounds inherent in GUI-driven processes.

Related Skills

Explore More Data & Analytics Skills→
MCERQUAMCERQUA
1

screaming-frog

Execute technical SEO audits with Screaming Frog SEO Spider headless CLI.

Community
Advanced
AgriciDanielAgriciDaniel
566

seo-firecrawl

Crawl and map websites to reveal pages, structure, and content gaps.

Community
Advanced
datasurfacedatasurface

setup mcp config

Create a .mcp.json file connecting Claude Code to a Kubernetes MCP server via SSE.

Official
Intermediate
REVREBELREVREBEL

technical-seo-checker

Identify and resolve technical SEO issues affecting crawlability and indexability.

Official
Advanced