skills.rest
CarbeneAI
CarbeneAICOMMUNITY
Explore all skills by this author →
Website𝕏@CarbeneAIGitHub

brightdata

Retrieve blocked or JavaScript-rendered web pages as markdown via tiered fetching.

9|1|Updated Apr 16, 2026
Search Tags:#web scraping#playwright#webfetch#curl#brightdata#captcha#progressive-escalation
One-click install
npx skills add https://github.com/CarbeneAI/Forge --skill brightdata-carbeneai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: brightdata
Source: https://github.com/CarbeneAI/Forge/tree/main/.claude/skills/BrightData
Command: npx skills add https://github.com/CarbeneAI/Forge --skill brightdata-carbeneai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill provides a reliable, cost-aware method to retrieve web page content that may be blocked, require JavaScript rendering, or be protected by bot detection and CAPTCHAs, removing the need for manual browser workarounds.

Core Features & Use Cases

  • Four-tier progressive escalation: attempts WebFetch, curl with Chrome-like headers, Playwright browser automation, then Bright Data MCP only when needed.
  • Cost and performance aware: favors fast, free methods first and escalates to paid services only as required.
  • Markdown output and reporting: returns extracted page content in markdown and reports which tier succeeded plus warnings.
  • Use Cases: research or archival scraping, extracting content from SPAs, recovering pages blocked by simple bot checks, and scraping sites with CAPTCHAs using Bright Data.

Quick Start

Ask the skill to "Scrape https://example.com and return the page content as markdown".

Frequently Asked Questions about brightdata

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a website that blocks my requests with bot detection?▼

Scraping a website that blocks requests requires a progressive escalation method: attempting standard WebFetch, then curl with browser headers, Playwright automation, and finally Bright Data MCP to bypass bot detection and CAPTCHAs.

What is the best way to extract content from single-page applications that require JavaScript rendering?▼

The best way to extract content from single-page applications requiring JavaScript rendering is using Playwright browser automation, which executes scripts to load dynamic content before extracting the HTML or markdown.

How do I retrieve web page content as markdown without paying for expensive scraping APIs?▼

To retrieve web page content as markdown cost-effectively, use a tiered fetching approach that prioritizes free methods like WebFetch and curl with Chrome-like headers, escalating to paid Bright Data MCP only when necessary.

Does web scraping with Playwright work on sites protected by CAPTCHAs?▼

Web scraping with Playwright does not natively bypass CAPTCHAs, but integrating Bright Data MCP as a final fallback tier allows you to scrape sites protected by CAPTCHAs and return the content as markdown.

Why does my curl web scraping fail on modern websites while browsers load the page fine?▼

Your curl web scraping fails on modern websites because they often use bot detection to block non-browser requests, requiring escalation to Playwright automation or Bright Data MCP to mimic browser behavior and bypass blocks.

Related Skills

Explore More Software Engineering Skills→
pynbj1001pynbj1001
4

BrightData

Scrape URLs through a four-tier workflow with automatic fallbacks and markdown output.

Community
Advanced
wwsunwwsun
1

defuddle

Extract clean markdown content and metadata from web pages using Defuddle CLI.

Community
Basic
eykdeykd

web-scraper

Convert web pages to Markdown using the url-to-markdown-cli-tool.

Community
Intermediate
iArsalanshahiArsalanshah
7

ghostfetch

Extract webpage content from protected sites using a stealthy headless browser.

Community
Intermediate