scrape-webpage

Scrape webpage content, extract metadata, and download images for AEM Edge Delivery Services.

Updated Mar 27, 2025

Search Tags:#web scraping #metadata extraction #aem #content migration #image download #html extraction

Ready to run

npx skills add https://github.com/jfoxx/residentportal --skill scrape-webpage-jfoxx

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: scrape-webpage
Source: https://github.com/jfoxx/residentportal/tree/main/.skills/scrape-webpage
Command: npx skills add https://github.com/jfoxx/residentportal --skill scrape-webpage-jfoxx

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires playwright, sharp, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the process of extracting content, metadata, and images from webpages, preparing them for import or migration into AEM Edge Delivery Services.

Core Features & Use Cases

Content Extraction: Captures cleaned HTML and downloads all images from a given URL.
Metadata Harvesting: Extracts important page metadata like title, description, Open Graph tags, and JSON-LD.
Use Case: When migrating a website to AEM, use this Skill to quickly gather all necessary assets and content from each source page, ensuring a smooth transition.

Quick Start

Use the scrape-webpage skill to analyze the webpage at https://example.com/about-us.

Frequently Asked Questions about scrape-webpage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract webpage content and images for AEM Edge Delivery Services migration?▼

To extract webpage content for AEM migration, this Skill scrapes the target URL, captures cleaned HTML, downloads all images locally, and returns an analysis JSON with paths and metadata for direct import into AEM Edge Delivery Services.

What webpage metadata can I extract during a content migration?▼

During webpage metadata extraction, this Skill harvests the page title, description, Open Graph tags, and JSON-LD structured data, returning them within an analysis JSON file alongside the cleaned HTML content.

Do I need Node.js and Playwright to scrape web content and download images?▼

Yes, you need Node.js, Playwright, and Sharp installed in your environment. Playwright drives the web scraping and image download processes, while Sharp handles the image processing required to prepare assets for AEM import.

What's the best way to automate downloading images from a webpage for content migration?▼

Automating image downloads for content migration is handled by Playwright, which navigates the target webpage, identifies image assets, downloads them locally, and uses Sharp for image processing before returning local paths in the analysis JSON.

Can I use this web scraping tool to get cleaned HTML instead of raw page source?▼

Yes, this web scraping tool extracts and returns cleaned HTML rather than the raw page source. It processes the target URL and outputs the cleaned HTML within an analysis JSON file ready for AEM Edge Delivery Services.