web-scraper

Fetch web pages and save them as HTML or Markdown with optional images.

47|4|Updated Feb 6, 2026

Search Tags:#markdown #html #offline #images #requests #web-scraping #beautifulsoup

Ready to run

npx skills add https://github.com/agentbay-ai/agentbay-skills --skill web-scraper-agentbay-ai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-scraper
Source: https://github.com/agentbay-ai/agentbay-skills/tree/main/web-scraper
Command: npx skills add https://github.com/agentbay-ai/agentbay-skills --skill web-scraper-agentbay-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests, beautifulsoup4, and includes scripts (resource) components.

What problem does it solve?

Automates saving and archiving web content locally by fetching pages and downloading embedded images for offline access and data collection.

Core Features & Use Cases

Single-page scraping to HTML or Markdown with optional image downloads.
Recursive crawling within the same domain to build offline archives and datasets.
Simple conversion to Markdown with embedded image references for documentation or research.

Quick Start

Provide a URL and desired output format, and the tool will fetch the page and save it locally.

Frequently Asked Questions about web-scraper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape a web page to Markdown with images for offline access?▼

You can scrape a web page to Markdown with images by fetching the content and saving it locally. This tool downloads embedded images and stores outputs in a structured directory for offline access.

Can I recursively crawl pages within the same domain using BeautifulSoup?▼

Yes, you can recursively crawl pages within the same domain using BeautifulSoup. The tool supports recursive scraping to build offline archives and datasets while respecting robots.txt by default.

Does requests support saving HTML content locally with embedded images?▼

Yes, requests supports fetching web page content to save HTML locally. The tool identifies and fetches pages, optionally downloading embedded images and storing them in a structured directory.

What is the best way to archive web content locally as HTML or Markdown?▼

The best way to archive web content locally as HTML or Markdown is automating the fetch process. This tool fetches pages and downloads embedded images, creating structured directories for data collection.

Does web scraping with requests respect robots.txt by default?▼

Yes, web scraping with requests respects robots.txt by default. The tool fetches web page content and saves it locally in HTML or Markdown format while adhering to standard crawling restrictions.