BrightData

Scrape web pages with four fallback tiers and output Markdown.

Updated Aug 23, 2026

Search Tags:#playwright #scraping #web #captcha #multi-tier #bright-data

Ready to run

npx skills add https://github.com/larsboes/pai-marketplace --skill brightdata-larsboes

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: BrightData
Source: https://github.com/larsboes/pai-marketplace/tree/main/marketplace/plugins/scraping/skills/BrightData
Command: npx skills add https://github.com/larsboes/pai-marketplace --skill brightdata-larsboes

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Progressive web scraping often fails due to bot detection, CAPTCHA, or JavaScript-heavy sites. This skill provides a four-tier, fallback-enabled workflow (WebFetch, customized Curl, Playwright browser automation, and Bright Data MCP) to reliably retrieve content from URLs.

Core Features & Use Cases

Tiered escalation: automatically fall back across tiers to handle bot protection, blocking, and dynamic content.
Single-URL scraping and multi-page crawling: fetch one page or map an entire site with link discovery.
Markdown output and structured results: deliver content in markdown with optional site maps and cost estimates.
Integration with Bright Data MCP for advanced scraping when other tiers fail.

Quick Start

Provide a URL and the skill will automatically attempt Tier 1, then Tier 2, Tier 3, and Tier 4 as needed to return the content in Markdown.

Frequently Asked Questions about BrightData

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape websites that block standard requests or use CAPTCHA?▼

Multi-tier web scraping handles blocked requests by automatically escalating through four fallback layers: WebFetch, customized curl, Playwright browser automation, and Bright Data MCP. This tiered approach reliably retrieves content from URLs protected by bot detection.

What's the best way to crawl multiple pages of a site with JavaScript-heavy content?▼

Multi-page crawling with fallback tiers retrieves dynamic JavaScript content by first trying WebFetch, then escalating to Playwright browser automation if needed. It maps site links and returns the fetched content structured in markdown format.

How does tiered fallback work for resilient web scraping?▼

Tiered fallback works by sequentially attempting four scraping methods: WebFetch first, then curl with Chrome headers, followed by Playwright browser automation, and finally Bright Data MCP. If a lower tier fails to retrieve content, the next tier automatically activates.

Do I need Bright Data MCP access to use this web scraping workflow?▼

Bright Data MCP access is only required for the fourth and final fallback tier. The workflow first attempts scraping using WebFetch, curl with Chrome headers, and Playwright browser automation before escalating to Bright Data MCP for advanced bot protection.

Can I customize the default scraping behavior for specific local environments?▼

You can override default scraping configurations by setting up an optional local customization directory. This allows you to tailor the WebFetch, curl, and Playwright tier behaviors to suit specific site requirements before escalating to Bright Data MCP.

Why does WebFetch fail on some sites and require Playwright browser automation?▼

WebFetch fails on sites using bot detection or heavy JavaScript rendering because it cannot execute dynamic scripts. When WebFetch and curl fail, the workflow escalates to Playwright browser automation to render the page and retrieve the actual content.