skills.rest
farzanmrz
farzanmrzCOMMUNITY
Explore all skills by this author →
GitHub

scrape

Fetch clean web content from URLs into markdown, HTML, JSON, or screenshots.

Updated Jul 18, 2026
Search Tags:#markdown#html#json#web content#pagination#scrape#bright data
One-click install
npx skills add https://github.com/farzanmrz/beat-radar-agent --skill scrape-farzanmrz
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrape
Source: https://github.com/farzanmrz/beat-radar-agent/tree/main/.claude/skills/scrape
Command: npx skills add https://github.com/farzanmrz/beat-radar-agent --skill scrape-farzanmrz

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the friction of collecting web content by turning one or many URLs into clean, usable output for analysis, extraction, or archiving.

Core Features & Use Cases

  • Clean content retrieval: Scrape a single page into markdown, HTML, JSON, or a screenshot.
  • Batch URL processing: Handle small lists or larger URL sets with safe parallelism and verification.
  • Pagination workflows: Crawl paginated listings page by page until the content ends.
  • Reliability checks: Detect block pages, verify non-empty results, and retry or escalate when needed.
  • Use case: A researcher can collect article pages from a publication, verify each result, and save them for downstream analysis.

Quick Start

Use the scrape skill to fetch the page at https://example.com/article as clean markdown and save it to a file.

Frequently Asked Questions about scrape

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape web content into clean markdown or JSON?▼

To scrape web content into clean markdown or JSON, you can fetch single pages or URL lists and convert them directly into markdown, HTML, JSON, or screenshots for downstream extraction and analysis.

What is the best way to scrape paginated listings until the content ends?▼

Scraping paginated listings is handled by crawling page by page until the content ends, ensuring you retrieve all available data without manual intervention for each page.

Does web scraping work with JS-heavy or geo-restricted sites?▼

Web scraping works with JS-heavy or geo-restricted sites by applying country targeting, block-page detection, and fallback handling to reliably retrieve content that normally requires verification.

Can I process a large batch of URLs for scraping in parallel?▼

Batch URL processing supports larger URL sets with safe parallelism and verification, ensuring non-empty results are collected efficiently without overwhelming the source server.

Why does my web scraping return empty results or block pages?▼

Empty results or block pages occur when access is restricted; the scraping process detects block pages, verifies non-empty results, and automatically retries or escalates to fallback handling.

Do I need Bright Data to scrape clean web content?▼

You need the Bright Data CLI to scrape clean web content, as it provides the required support for markdown, HTML, JSON, screenshot output, country targeting, and block-page detection.

Related Skills

Explore More Software Engineering Skills→
Gull-StackGull-Stack

social-media

Automate social media strategy, content planning, and engagement optimization across platforms.

Official
Basic
sridsrid
13

article-extractor

Extract clean article content from URLs and output Markdown to stdout.

Community
Intermediate
Amazon-QuickAmazon-Quick
47

browser-data-scraper

Extract structured data from websites via browser automation with JSONL storage.

Official
Advanced
zhayujiezhayujie
46.4k

web-fetch

Fetch web page HTML and extract title and readable text.

Community
Basic