Scraping

The skill escalates through four tiers automatically: WebFetch, curl with Chrome browser headers, Playwright browser automation, and finally the Bright Data proxy service. Each tier handles progressively stronger defenses, including CAPTCHAs and JavaScript-rendered pages.

Use the code-first actor wrappers such as scrapeInstagramProfile, scrapeTikTokHashtag, or scrapeLinkedInProfile with an APIFY_TOKEN set in the environment. Results are filtered in TypeScript before reaching the model, reducing token usage by up to 99% compared to MCP calls.

Light Crawl handles up to 50 pages using batch scraping with iterative link discovery, suited for site sections. Full Crawl uses the Bright Data Crawl API for entire sites at roughly $1.50 per 1,000 pages, with progress polling and structured site map output.

Yes, Tier 3 uses Playwright browser automation to fully render JavaScript, execute dynamic content loading, and handle cookies or sessions. If browser automation is blocked by CAPTCHAs, the workflow escalates to the Bright Data residential proxy tier.

MCP calls return unfiltered datasets of around 50,000 tokens into model context, while code-first scripts filter results in the execution environment and return only relevant items, typically around 500 tokens. This yields 90-99% token savings and faster execution.

All four tiers can fail when a site requires login authentication, is behind a paywall, has geographic restrictions, or is completely down. Tier 4 also consumes Bright Data credits, so cheaper tiers are always attempted first unless Bright Data is explicitly requested.