skills.rest
arbory-digital-inc
arbory-digital-incOFFICIAL
Explore all skills by this author →
Website𝕏@arborydigitalGitHub

scrape-webpage

Extract webpage content, metadata, and images into local assets.

3|Updated Apr 17, 2025
Search Tags:#html#migration#metadata#playwright#webpage#scrape#image-mapping
One-click install
npx skills add https://github.com/arbory-digital-inc/sas-demo --skill scrape-webpage-arbory-digital-inc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrape-webpage
Source: https://github.com/arbory-digital-inc/sas-demo/tree/main/.claude/skills/scrape-webpage
Command: npx skills add https://github.com/arbory-digital-inc/sas-demo --skill scrape-webpage-arbory-digital-inc

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires playwright, sharp, and includes scripts (resource) components.

What problem does it solve?

This Skill automates the extraction of content, metadata, and images from a live webpage to support migration workflows, producing local asset mappings and structured results.

Core Features & Use Cases

  • Webpage analysis: fetches title, description, Open Graph, JSON-LD, and canonical data.
  • Content extraction: returns cleaned HTML with essential attributes and local image references.
  • Asset mapping: downloads images, converts to PNG when needed, and provides an image map for migration.

Quick Start

Use the scrape-webpage skill to analyze a URL and generate an output directory containing metadata.json, cleaned.html, a screenshot, and an images/ folder for local assets. Then pass the results to the next step (identify-page-structure).

Frequently Asked Questions about scrape-webpage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape webpage content and images for migration?▼

To scrape webpage content and images for migration, you can use a browser automation tool to extract the HTML, download local image assets, and generate a metadata.json file mapping the assets.

How do I extract metadata and Open Graph data from a live webpage?▼

Extracting metadata and Open Graph data from a live webpage involves fetching the URL with browser automation to capture the title, description, JSON-LD, and canonical links into a structured JSON format.

Do I need Playwright to extract HTML and download images from a webpage?▼

Yes, you need Node.js with Playwright to automate the browser for full-page analysis and image downloading, while Sharp is used for image processing to convert assets to PNG when needed.

What is the best way to map downloaded images during a webpage migration?▼

The best way to map downloaded images during webpage migration is to save them to a local images folder and generate an image map, ensuring cleaned HTML references the local assets correctly.

Can I use scraped webpage HTML directly in a page-import workflow?▼

Yes, you can use the scraped webpage HTML directly in a page-import workflow because the process returns cleaned HTML with essential attributes and local image references ready for structural analysis.

What are the limitations of using browser automation for web scraping?▼

Limitations of using browser automation for web scraping include requiring a Node.js environment with specific dependencies like Playwright and Sharp, and the process may struggle with highly dynamic JavaScript content.

Related Skills

Explore More Content & Communication Skills→
paolomozpaolomoz

scrape-webpage

Scrapes webpages and extracts metadata, images, and cleaned HTML for migration.

Community
Advanced
robomotioniorobomotionio
2

web-scraper

Extract structured data from HTML using CSS selectors or XPath queries.

Official
Intermediate
haboshihaboshi

svg-to-webp

Convert SVG images to WebP or PNG using the sharp library.

Community
Intermediate
shepherdjerredshepherdjerred
127

lightpanda-browser

Fetch web content and execute JavaScript with a Zig-based headless browser.

Community
Intermediate