web-data-acquisition

Bypassing Cloudflare during web scraping is handled by routing the request through a structured escalation path, starting with a standard fetch and advancing to local browser-based tools only if blocked. This prevents improvising tools and ensures reliable data retrieval.

Extracting data from JavaScript-rendered web pages is managed by routing the task to specialized local stack tools like crawl4ai or scrapegraph-ai. This approach assesses site complexity and applies the appropriate browser-based tool to handle JS-heavy rendering challenges.

Crawling a documentation site and exporting content to markdown is executed by initiating a web-data acquisition task. The skill routes the crawling process to the most efficient local tool, extracting structured content and formatting it into markdown for immediate use.

Yes, a defined hierarchy of local tools including crawl4ai, katana, scrapegraph-ai, and camofox is required. This skill relies on this pre-configured local stack to route tasks efficiently and ensure reliable data retrieval across varying site complexities.

Web automation scripts often fail on complex sites due to inconsistent tool selection and unhandled anti-bot blocks. This skill solves this by providing a standardized routing layer that safely escalates tasks to the correct local tool for blocked content or JS-heavy rendering.