skills.rest
cascade-protocol
cascade-protocolOFFICIAL
Explore all skills by this author →
Website𝕏@cascade_fyiGitHub

web

Crawl URLs with dynamic rendering and anti-bot bypass via paid API.

14|1|Updated Feb 18, 2026
Search Tags:#web search#web scraping#exa#crawling#micropayments#anti-bot
One-click install
npx skills add https://github.com/cascade-protocol/agentbox --skill web-cascade-protocol
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web
Source: https://github.com/cascade-protocol/agentbox/tree/main/skills/agentbox-web
Command: npx skills add https://github.com/cascade-protocol/agentbox --skill web-cascade-protocol

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a robust solution for web scraping and searching, overcoming limitations of standard web fetchers by handling dynamic content, anti-bot measures, and bulk requests.

Core Features & Use Cases

  • Advanced Web Crawling: Fetches content from any URL, escalating through HTTP, headless browser, and stealth browser with proxy to bypass blocks.
  • Bulk Crawling: Efficiently processes multiple URLs in a single request.
  • Web Search: Leverages Exa for comprehensive web search results.
  • Automated Payments: Integrates with x_payment for seamless USDC micropayments for each crawl or search.
  • Use Case: When a standard web_fetch fails due to JavaScript rendering or anti-bot protection, use this Skill to reliably extract the content, or to search for specific information across the web.

Quick Start

Use the web skill to crawl the URL https://example.com and return the content as markdown.

Frequently Asked Questions about web

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape JavaScript-rendered web pages when standard fetch fails?▼

Web scraping JavaScript-rendered pages requires escalating through HTTP, headless browser, and stealth browser with proxy layers to bypass anti-bot blocks and extract dynamically loaded content as markdown.

What is the best way to crawl multiple URLs at once for bulk content extraction?▼

Bulk web crawling processes multiple URLs in a single request to extract content efficiently, supporting format options like markdown and CSS selector extraction for structured data retrieval.

How does automated USDC micropayment integration work for web scraping APIs?▼

Automated USDC micropayments integrate via x_payment to handle per-crawl or per-search transactions seamlessly, enabling access to paid web scraping and search APIs without manual billing.

Can I use Exa web search to find specific information across the web?▼

Exa web search leverages a paid API to perform comprehensive web search queries, returning relevant results for specific information retrieval tasks across the web.

Does web scraping with anti-bot bypass support CSS selector extraction?▼

Anti-bot bypass web scraping supports CSS selector extraction, allowing targeted data extraction from protected pages while escalating through stealth browser proxies to ensure access.

When should I use a stealth browser with proxy for web crawling instead of a standard fetch?▼

Use a stealth browser with proxy for web crawling when standard HTTP fetches fail due to anti-bot protection or JavaScript rendering, ensuring reliable content extraction from complex pages.

Related Skills

Explore More Software Engineering Skills→
cascade-protocolcascade-protocol
14

agentbox-twitter

Search tweets, fetch tweet context, and retrieve user profiles via paid Twitter API.

Official
Intermediate
UniquecreteUniquecrete

web-scraper

Extract structured data from web pages using CSS schemas and crawling.

Community
Intermediate
rongarederongarede
4

web-fetch-fallback

Route web content retrieval to alternative fetch methods based on URL type.

Community
Intermediate
vm0-aivm0-ai
76

scrapeninja

Extract data from anti-bot-protected websites via a scraping API.

Official
Intermediate