tavily-crawl

Crawl websites and export each page as a markdown file using the Tavily CLI.

1|Updated May 10, 2026

Search Tags:#markdown #cli #offline #web #crawl #tavily

Ready to run

npx skills add https://github.com/a2ajinkya/phone-pi --skill tavily-crawl-a2ajinkya

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: tavily-crawl
Source: https://github.com/a2ajinkya/phone-pi/tree/main/skills/tavily-crawl
Command: npx skills add https://github.com/a2ajinkya/phone-pi --skill tavily-crawl-a2ajinkya

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill enables automated crawling of a website to extract content from multiple pages and save each page as a local markdown file for offline access and documentation.

Core Features & Use Cases

Site-wide crawl with depth and breadth controls to fetch sections like /docs or /api.
Markdown export: saves each crawled page as an individual markdown file for easy offline access and documentation.
Targeted extraction: supports path filtering and semantic focus to pull relevant content across large sites.

Quick Start

Run tvly crawl 'https://docs.example.com' to fetch pages and save each page as local markdown.

Frequently Asked Questions about tavily-crawl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I crawl a website and save pages as local markdown files?▼

Website crawling for markdown export extracts content from multiple site pages and saves each as an individual markdown file for offline access. It uses depth and breadth controls to fetch specific sections like /docs or /api.

Can I limit a web crawl to specific paths like /docs or /api?▼

You can limit a web crawl to specific paths like /docs or /api using path filtering controls. This targets relevant sections across large sites and prevents fetching irrelevant pages, keeping the markdown output focused.

How do I control crawl depth and breadth when extracting site content?▼

Control crawl depth and breadth by setting traversal parameters before running the crawl command. Depth limits how many links deep the crawler follows, while breadth limits how many pages per level are fetched and exported as markdown.

What is the best way to collect documentation across a site for offline research?▼

The best way to collect documentation for offline research is a targeted site crawl with semantic focus and path filtering. This extracts guides or product pages and exports them as per-page markdown files for local content ingestion.

Does the Tavily crawl CLI require any dependencies to save pages as markdown?▼

The Tavily crawl CLI has no listed dependencies, allowing you to directly run the crawl command to fetch website content. It processes extracted pages and exports them directly as local markdown files without external packages.