crawl4ai-skill
Scrape JavaScript-heavy sites and extract structured web data
All Skills in This Repository (1)
Pure Emerald Level IndicatorsFrequently Asked Questions
FAQPage SchemaHow to install crawl4ai-skill?โผ
Run `npx skills add brettdavies/crawl4ai-skill --all -g -y` in your terminal to install the skill globally. You also need the Crawl4AI Python library installed via `pip install crawl4ai` followed by `crawl4ai-setup`.
How to scrape JavaScript-heavy websites?โผ
This skill renders pages in a headless browser with network-idle waiting, so content loaded by JavaScript is fully captured. A single command like `crwl <url> -o markdown` returns clean markdown.
How to extract structured data without paying LLM costs?โผ
Generate a reusable CSS schema once with the bundled script, then apply it to any number of pages for fast, deterministic extraction with zero per-request LLM cost.
Does crawl4ai-skill work with Claude Code and Cursor?โผ
Yes. It follows the universal SKILL.md standard and loads in Claude Code, Codex, Cursor, OpenCode, Cline, and other compatible hosts.
Can it crawl many URLs or handle logins?โผ
Yes. Batch scripts crawl multiple URLs concurrently, and session templates support authenticated, multi-step flows with proxies and anti-detection options.
Related Repositories in Software Engineering
View All in Software Engineeringโopenclaw
Run a personal AI assistant across your devices and chat apps
superpowers
Gives coding agents a disciplined workflow from idea to merged code
react
AI agent skills for building, testing, and porting React core