Crawl websites and extract content into Markdown, JSON, or text.
npx skills add https://github.com/abdullah1854/ClaudeSuperSkills --skill crawl-cli
Please help me install this Agent Skill. Skill: crawl-cli Source: https://github.com/abdullah1854/ClaudeSuperSkills/tree/main/crawl-cli Command: npx skills add https://github.com/abdullah1854/ClaudeSuperSkills --skill crawl-cli
Provides a CLI to crawl websites and output clean Markdown or JSON.
Crawl a documentation page and convert to Markdown.
High-intent search queries and answers about installing and using this skill.
Web crawling extracts content from websites and converts it to Markdown format. This Skill handles single pages and multi-page traversal, respecting robots.txt, parsing URLs, and outputting clean Markdown with metadata like title, source URL, and timestamp.
Yes, this Skill supports multi-page traversal with configurable depth. It crawls paginated sites, extracts content across multiple pages, and outputs structured results with pagination support in Markdown, JSON, or text format.
Web crawling with this Skill produces three output formats: Markdown, JSON, or plain text. Each format includes structured metadata like title, source URL, timestamp, and extracted links for documentation sites, blogs, and knowledge bases.
Yes, this Skill is designed for documentation sites, blogs, and knowledge bases. It extracts clean content, handles both single-page and multi-page structures, and respects robots.txt to ensure compliant crawling.
Web crawling handles both single pages and multi-page traversal. You can configure crawl depth, extract content from individual pages, or traverse paginated structures to produce structured Markdown or JSON output with all discovered links.
Web crawling extracts and preserves links, titles, source URLs, and timestamps in the output. Metadata is included in structured results, enabling you to reference the crawl source and navigate discovered pages programmatically.
seohowAnalyze robots.txt directives, URL status codes, redirect chains, and crawl depth.
smallnestConvert PDFs, Word, Excel, HTML, and images with OCR into Markdown.
sadie100Generate and validate llms.txt files from crawled website content.
archibateScrapes and extracts data from JavaScript-rendered and bot-protected websites.
