skills.rest
pynbj1001
pynbj1001COMMUNITY
Explore all skills by this author →
GitHub

BrightData

Scrape URLs through a four-tier workflow with automatic fallbacks and markdown output.

4|1|Updated Mar 25, 2026
Search Tags:#automation#playwright#curl#web-scraping#brightdata#markdown-output#tiered-scraping
One-click install
npx skills add https://github.com/pynbj1001/alpha-sense --skill brightdata-pynbj1001
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: BrightData
Source: https://github.com/pynbj1001/alpha-sense/tree/main/.pai_runtime/.claude/skills/BrightData
Command: npx skills add https://github.com/pynbj1001/alpha-sense --skill brightdata-pynbj1001

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates reliable retrieval of web content from URLs, even when pages are dynamic or protected by bot detection.

Core Features & Use Cases

  • Four-tier scraping workflow: WebFetch, customized Curl, Playwright browser automation, and Bright Data MCP with automatic fallbacks.
  • Markdown output of retrieved content with context about which tier succeeded.
  • Handles both simple and JS-heavy sites, CAPTCHA, and access restrictions.
  • Use Case: robust data gathering for research, competitive intelligence, and automation pipelines.

Quick Start

Provide a URL to fetch and the skill will escalate through all four tiers automatically.

Frequently Asked Questions about BrightData

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape dynamic websites that require JavaScript rendering and block standard requests?▼

Scraping dynamic websites with JavaScript rendering requirements is handled through an automated four-tier workflow. It escalates from basic WebFetch to customized Curl, then Playwright browser automation, and finally Bright Data MCP to bypass bot detection and retrieve content.

What is the best way to bypass bot detection and CAPTCHA when retrieving web content?▼

Bypassing bot detection and CAPTCHA is achieved by utilizing a tiered scraping fallback architecture. When standard automated requests fail, the system automatically escalates to Playwright browser automation and Bright Data MCP to reliably fetch protected web pages.

Can I use Playwright and Curl for automated web scraping in a fallback workflow?▼

Playwright and customized Curl are integrated into an automated tiered scraping workflow. This architecture applies progressive fallbacks across WebFetch, Curl, and Playwright browser automation to reliably retrieve web content from public pages and dynamic sites.

How do I extract web content as markdown from pages with access restrictions?▼

Extracting web content as markdown from restricted pages is automated through a progressive four-tier scraping workflow. It applies fallbacks across WebFetch, Curl, Playwright, and Bright Data MCP, outputting retrieved content as markdown with context about the successful tier.

Why does web scraping fail on certain sites and how does tiered scraping handle it?▼

Web scraping fails when sites implement bot detection or JavaScript rendering requirements. Tiered scraping handles this by automatically escalating from simple WebFetch to Playwright browser automation and Bright Data MCP, applying progressive fallbacks until content is retrieved.

Do I need to configure Bright Data MCP manually to scrape sites with CAPTCHA?▼

Configuring Bright Data MCP manually is not required to scrape sites with CAPTCHA. The skill automates the entire four-tier workflow, applying Bright Data MCP as a final fallback only when WebFetch, customized Curl, and Playwright browser automation fail to retrieve content.

Related Skills

Explore More Data & Analytics Skills→
verrio1verrio1

BrightData

Retrieve web page content from URLs using a four-tier scraping workflow.

Community
Advanced
polymerdaopolymerdao
1

polymer-pay-riveter

Automate web search, scraping, and data extraction through a Polymer Pay proxy.

Official
Intermediate
parcadeiparcadei
3.9k

firecrawl-scrape

Scrape web pages and extract content via Firecrawl MCP into Markdown, HTML, or plain text.

Community
Intermediate
olostepolostep

batch

Scrape up to 10,000 URLs in parallel with configurable output formats.

Community
Intermediate