Crawl websites and export each page as a markdown file using the Tavily CLI.
npx skills add https://github.com/a2ajinkya/phone-pi --skill tavily-crawl-a2ajinkya
Please help me install this Agent Skill. Skill: tavily-crawl Source: https://github.com/a2ajinkya/phone-pi/tree/main/skills/tavily-crawl Command: npx skills add https://github.com/a2ajinkya/phone-pi --skill tavily-crawl-a2ajinkya
This Skill enables automated crawling of a website to extract content from multiple pages and save each page as a local markdown file for offline access and documentation.
Run tvly crawl 'https://docs.example.com' to fetch pages and save each page as local markdown.
High-intent search queries and answers about installing and using this skill.
Website crawling for markdown export extracts content from multiple site pages and saves each as an individual markdown file for offline access. It uses depth and breadth controls to fetch specific sections like /docs or /api.
You can limit a web crawl to specific paths like /docs or /api using path filtering controls. This targets relevant sections across large sites and prevents fetching irrelevant pages, keeping the markdown output focused.
Control crawl depth and breadth by setting traversal parameters before running the crawl command. Depth limits how many links deep the crawler follows, while breadth limits how many pages per level are fetched and exported as markdown.
The best way to collect documentation for offline research is a targeted site crawl with semantic focus and path filtering. This extracts guides or product pages and exports them as per-page markdown files for local content ingestion.
The Tavily crawl CLI has no listed dependencies, allowing you to directly run the crawl command to fetch website content. It processes extracted pages and exports them directly as local markdown files without external packages.
robomotionioAutomate web search, content extraction, and site mapping with Tavily.
zzafergokConvert Markdown documents to HTML using marked.js, pandoc, gomarkdown, or static site generators.
zbyExtract structured knowledge and actionable insights into atomic notes.
xberg-ioScrape webpages and convert HTML content to Markdown.
