skills.rest
AtlasOmnia
AtlasOmniaCOMMUNITY
Explore all skills by this author →
Website𝕏@AtlasOmniaGitHub

dynamic-content-extraction

Extract structured data from JavaScript-heavy sites using browser snapshots and TreeWalker text-node traversal.

115|9|Updated Aug 5, 2026
Search Tags:#data extraction#react#browser automation#web scraping#lazy loading#price extraction#treewalker
One-click install
npx skills add https://github.com/AtlasOmnia/donna-starter --skill dynamic-content-extraction-atlasomnia
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dynamic-content-extraction
Source: https://github.com/AtlasOmnia/donna-starter/tree/main/skills/research/dynamic-content-extraction
Command: npx skills add https://github.com/AtlasOmnia/donna-starter --skill dynamic-content-extraction-atlasomnia

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Prices, listings, and key fields on React/Vue/Angular sites often never appear in standard accessibility snapshots or web extraction tools, leaving scrapers empty-handed even though the data is visible on screen. ## Core Features & Use Cases - Full accessibility snapshots: Uses browser_snapshot(full=true) to surface dynamically injected text that compact mode omits. - TreeWalker text-node extraction: Recovers React split-text values (e.g., price and currency in sibling spans) by walking leaf text nodes and climbing to card containers. - Lazy-load and API handling: Scrolls to trigger rendering, then discovers same-origin frontend data APIs via performance entries and in-console fetch calls. - Use Case: Extract hotel names and nightly prices from an IHG or Booking.com results page where web_extract times out and the compact snapshot shows no prices. ## Quick Start Extract all hotel names and prices from this open booking site results page, scrolling to load every listing first.

Frequently Asked Questions about dynamic-content-extraction

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract prices from JavaScript-heavy websites?▼

Take a full accessibility snapshot with browser_snapshot(full=true), scroll several times to trigger lazy loading, then run a TreeWalker script in the browser console that matches price patterns in leaf text nodes and walks up to the containing card for related fields.

Why does web_extract fail on React or SPA sites?▼

Single-page applications render content client-side, so static fetchers time out or return empty content. Instead of retrying, drive a real browser session and extract data through console-evaluated JavaScript against the live DOM.

How do I handle lazy-loaded listings when scraping?▼

Scroll down four to six times and wait two to three seconds for the framework to hydrate new cards before extracting. Repeat the scroll-wait-extract loop and accept partial coverage, reporting how many of the total results were captured.

Can I use a vision model to read prices from screenshots?▼

Vision models are unreliable for exact numeric data; they misread prices and conflate similar names. Use them only as a last resort for layout questions, and extract numbers from DOM text nodes instead.

What if direct API calls to a booking site are blocked?▼

CDN and WAF protections like Akamai or Cloudflare block curl requests. Instead, reproduce the site's own frontend API calls with fetch() inside the authenticated browser console, discovering endpoints via performance.getEntriesByType('resource').

Related Skills

Explore More Data & Analytics Skills→
browser-actbrowser-act
5.5k

walmart-keyword-search

Extract paginated product listings from Walmart keyword search result pages.

Official
Intermediate
nexscope-ainexscope-ai
510

amazon-competitor-analysis

Compare Amazon listings, reviews, pricing, and ads to generate actionable gap reports.

Official
Advanced
kortix-aikortix-ai
20.2k

competitor-diff

Fetches competitor pages, diffs them against stored snapshots, and posts meaningful changes to Slack.

Official
Intermediate
zenbuappszenbuapps

aibdd.auto.ts.it.handlers.readmodel-then

Verify rendered UI content in React IT Gherkin tests using Testing Library screen queries.

Official
Intermediate