obscura

Render JavaScript-heavy pages in a headless browser and extract text, HTML, links, or JSON.

5|Updated Aug 8, 2016

Search Tags:#browser automation #web scraping #headless browser #javascript rendering #stealth mode #dom extraction

Ready to run

npx skills add https://github.com/gbencke/dotfiles --skill obscura

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: obscura
Source: https://github.com/gbencke/dotfiles/tree/main/pi/skills/obscura
Command: npx skills add https://github.com/gbencke/dotfiles --skill obscura

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you reliably extract content from web pages and automate browser interactions when sites render dynamic JavaScript or use bot protections.

Core Features & Use Cases

Headless Fetching: Render a page and extract outputs like title, links, HTML, or plain text for downstream analysis.
Dynamic Content Handling: Wait for specific conditions (e.g., network idle or a selector) before extracting results.
Stealth Scraping: Reduce detection by using stealth mode to better handle common anti-bot systems.
Parallel Extraction: Scrape multiple URLs concurrently and evaluate DOM expressions to produce structured results.

Quick Start

Run: obscura fetch https://example.com --quiet --dump text

Frequently Asked Questions about obscura

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scrape JavaScript-rendered web pages?▼

To scrape JavaScript-rendered web pages, you need a headless browser to execute the dynamic scripts and extract the fully rendered DOM. This approach waits for network idle conditions or specific selectors before capturing the final page content.

What is the best way to extract structured data from multiple URLs concurrently?▼

The best way to extract structured data from multiple URLs is through parallel multi-URL scraping, which fetches pages concurrently and evaluates DOM expressions using selectors to produce structured JSON outputs for downstream analysis.

Does stealth mode work against anti-bot detection systems?▼

Stealth mode helps reduce detection by common anti-bot systems during headless browser automation. It masks automated browser fingerprints, allowing web scraping tasks to better handle sites with basic bot protections.

Can I wait for a specific DOM selector before extracting web content?▼

Yes, you can wait for a specific DOM selector before extracting web content. The headless browser supports dynamic waiting conditions, allowing scraping to pause until the required element renders on the page.

How do I get plain text or HTML output from a headless browser fetch?▼

You can get plain text or HTML output from a headless browser fetch by specifying the desired output format. The extraction supports dumping the rendered page as text, raw HTML, extracted links, or structured JSON.