Extract and structure web page content into JSON with metadata.
npx skills add https://github.com/guifav/openclaw-skills --skill web-scraper-guifav
Please help me install this Agent Skill. Skill: web-scraper Source: https://github.com/guifav/openclaw-skills/tree/main/web-scraper Command: npx skills add https://github.com/guifav/openclaw-skills --skill web-scraper-guifav
Extracts and structures content from web pages by applying a multi‑strategy cascade that detects articles, scrapes text, and gathers metadata.
Provide a target URL and let the skill fetch, clean, and output structured article data including metadata.
High-intent search queries and answers about installing and using this skill.
To extract text from JavaScript-rendered web pages, this tool applies a multi-strategy cascade using browser automation alongside static HTML parsing. It reliably retrieves full page text and metadata even when content loads dynamically.
The best way to scrape article content and remove boilerplate is using a multi-strategy cascade with news/article detection. It identifies publishable content via URL patterns and Schema.org metadata, stripping overlays to output clean text.
Web scraping works on paywalled pages by applying specific hard and soft paywall handling strategies. It safely exposes content and removes overlays when permissible, allowing extraction of available text without bypassing restrictions.
Yes, you can extract structured metadata like authors, dates, categories, and keywords from web pages. The output is JSON-formatted with provenance and data quality metrics, offering clean text alongside comprehensive article metadata.
You can configure custom CSS selectors for content extraction using declarative YAML configuration files. This allows you to define specific extraction rules and selectors to tailor the scraping pipeline to your target domains.
To extract entities like people, organizations, and locations from scraped articles, enable optional LLM-based entity extraction. This feature identifies relationships and structured data directly from the cleaned text during the scraping process.
vuongtu1k9-ship-itExtract web page content into structured JSON with configurable YAML pipelines.
profmikegreeneParses pilot plans and hypotheses into structured JSON or tabular data for downstream analysis.
icyzhScrape static and JavaScript-rendered web pages using stealth browser automation.
zakaroooExtract web page content and metadata via the page_reader function.
