Initiate, monitor, and export website crawls via the Cloudflare Browser Rendering API.
npx skills add https://github.com/jscraik/Agent-Skills --skill cf-crawl
Please help me install this Agent Skill. Skill: cf-crawl Source: https://github.com/jscraik/Agent-Skills/tree/main/utilities/cf-crawl Command: npx skills add https://github.com/jscraik/Agent-Skills --skill cf-crawl
This Skill automates the process of crawling websites using Cloudflare's Browser Rendering service, allowing you to capture web content in various formats and manage crawl jobs efficiently.
Crawl https://developers.cloudflare.com/workers/ to markdown with a 25-page limit and save the results under .crawl-output/workers.
High-intent search queries and answers about installing and using this skill.
To crawl a website and export Markdown, initiate a job targeting a specific URL, monitor its progress, and retrieve the results as per-page files or a merged digest. This process captures web content for local analysis and archiving.
The Cloudflare Browser Rendering API handles web scraping by initiating crawl jobs with customizable limits, depth, and rendering options. You monitor job progress, export results in JSON or Markdown, and cancel active jobs when necessary.
Yes, you need Cloudflare API credentials to start a crawl job. The crawling process requires specific parameters like target URL, page limits, and output formats to properly ingest web content via the Browser Rendering service.
Yes, you can monitor the status of an active web crawl and cancel it. The system handles administrative tasks for ingesting web content, allowing you to check the progress of existing crawl jobs and terminate them when necessary.
The best way to export crawled web pages into JSON format is to configure JSON as your output format when initiating the crawl. You retrieve the results locally as individual per-page files or as a single merged digest file.
Limitations when using Cloudflare Browser Rendering for web scraping include managing target URLs and page limits. You must monitor job status and potentially cancel active crawl jobs to avoid exceeding your specified administrative parameters and ingestion limits.
abdullah1854Crawl websites and extract content into Markdown, JSON, or text.
spencerbeggsGenerate a markdown table of contents from documentation file headers.
while-basicExtract web page content as Markdown or structured data via the Firecrawl API.
tedtv1007-ctrlFetch rendered HTML from dynamic websites via the Cloudflare Browser Rendering API.
