skills.rest
AstraBert
AstraBertCOMMUNITY
Explore all skills by this author →
Website𝕏@itscleliaGitHub

web-scraping

Scrape web page content into Markdown using the scpr CLI.

12|3|Updated Jan 29, 2026
Search Tags:#automation#markdown#cli#web-scraping#web-content#scpr#recursive-scraping
One-click install
npx skills add https://github.com/AstraBert/scpr --skill web-scraping-astrabert
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraping
Source: https://github.com/AstraBert/scpr/tree/main/.claude/skills/web-scraping
Command: npx skills add https://github.com/AstraBert/scpr --skill web-scraping-astrabert

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Scrape web pages efficiently by turning page content into Markdown using a simple CLI, eliminating manual copying from websites.

Core Features & Use Cases

  • Single-page scraping: extract a page's content to Markdown with a single URL.
  • Recursive scraping: follow links within the same domain to collect related pages.
  • Parallel scraping: speed up scraping with configurable concurrency.
  • Output & options: control output path, allowed domains, and depth with standard flags.

Quick Start

Ask the CLI to scrape a page by URL and save the results to a local folder.

Frequently Asked Questions about web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape web page content into Markdown from a URL?▼

To scrape web page content into Markdown, provide the target URL to the CLI using the --url flag. The tool extracts the page content and converts it into Markdown format, saving the results to a specified local output folder.

Can I recursively scrape linked pages within the same domain?▼

Yes, recursive scraping follows links within the same domain to collect related pages. Enable this mode with the --recursive flag, use --max to control scraping depth, and apply --allowed to restrict the crawling scope to specific domains.

How does parallel scraping work for extracting web content?▼

Parallel scraping speeds up web content extraction by processing multiple pages concurrently. Configure the concurrency level using the --parallel flag alongside the standard URL and output arguments to accelerate the scraping workflow.

Do I need to install any dependencies to use the scpr CLI for web scraping?▼

Yes, the scpr CLI is required as an external dependency to execute the scraping commands. Once the CLI is installed and accessible, you can pass URLs, set output paths, and configure recursive or parallel scraping modes.

What is the best way to control output paths and allowed domains when scraping websites?▼

Control output paths and allowed domains by using the --output flag to define the local save directory and the --allowed flag to specify permitted domains. This ensures scraped Markdown content is organized and crawling stays within boundaries.

Why scrape web pages into Markdown instead of copying content manually?▼

Scraping web pages into Markdown eliminates manual copying from websites by automatically extracting and formatting content. This provides clean, structured Markdown files for downstream processing, saving time and reducing formatting errors.

Related Skills

Explore More Software Engineering Skills→
dioerdendioerden

tmux

Execute keystrokes and capture output from tmux sessions.

Community
Intermediate
Blake-JohnBlake-John

pymupdf

Convert PDF documents into Markdown, JSON, and text for LLM workflows.

Community
Intermediate
LayerDynamicsLayerDynamics

mcp-cron-automation

Automates Trellio board operations via scheduled cron jobs and AI task execution using MCP-CRON tools.

Community
Advanced
meytalp-devmeytalp-dev

agent-browser

Automates browser navigation, form filling, and data extraction via CLI commands.

Community
Advanced