skills.rest
MathiasPaulenko
MathiasPaulenkoCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

WaveXis Web Scraping

Automates web scraping and site crawling with browser-based stealth.

Updated Jun 3, 2026
Search Tags:#automation#scraping#extraction#crawler#web#stealth
One-click install
npx skills add https://github.com/MathiasPaulenko/ai-toolkit --skill wavexis-web-scraping
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: WaveXis Web Scraping
Source: https://github.com/MathiasPaulenko/ai-toolkit/tree/main/skills/wavexis-web-scraping
Command: npx skills add https://github.com/MathiasPaulenko/ai-toolkit --skill wavexis-web-scraping

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and assets (resource) components.

What problem does it solve?

This skill solves the challenge of extracting data from complex, dynamic, or anti-bot protected websites by providing a robust CLI wrapper for browser automation.

Core Features & Use Cases

  • Stealth Mode: Bypasses common anti-bot detection by spoofing browser fingerprints and hiding automation indicators.
  • Multi-Format Extraction: Supports scraping content into Markdown, HTML, JSON, or plain text using CSS, XPath, or natural language selectors.
  • Complex Workflow Automation: Handles Shadow DOM content, authenticated sessions, and site-wide crawling with rate limiting and action caching.

Quick Start

Use the wavexis scrape command to extract the main article content from the target URL while enabling stealth mode to avoid detection.

Frequently Asked Questions about WaveXis Web Scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape data from a website that uses anti-bot detection?▼

Web scraping with anti-bot detection is handled using stealth mode, which spoofs browser fingerprints and hides automation indicators to bypass common protections. This allows automated extraction from protected sites without triggering blocks.

Can I extract content from Shadow DOM structures during web crawling?▼

Shadow DOM content extraction is supported during web crawling alongside standard DOM structures. The skill handles complex page elements and authenticated sessions, enabling comprehensive data retrieval from dynamic single-page applications.

What output formats can I get when scraping web pages with natural language selectors?▼

Scraping output formats include Markdown, HTML, JSON, and plain text. Selectors for extraction can be defined using CSS, XPath, or natural language, with the latter requiring optional API keys for processing.

Do I need a specific browser environment to automate web scraping and bypass anti-bot measures?▼

A configured environment with Chrome or Edge is required to run browser-based automation for web scraping. Optional API keys are also needed if you want to use natural language processing for element selection.

What is the best way to crawl an entire site with rate limiting and action caching?▼

Site-wide crawling is automated with built-in rate limiting and action caching to manage server load and improve efficiency. This approach ensures comprehensive data extraction across multiple pages while respecting target site resources.

Why does my automated web scraper fail on authenticated pages?▼

Automated scraping of authenticated pages requires handling complex session states. This skill facilitates extraction from authenticated sessions by maintaining browser context, ensuring data retrieval succeeds behind login walls.

Related Skills

Explore More Software Engineering Skills→
novskidevnovskidev
2

browser

Automate Chrome browser actions via the Chrome DevTools Protocol using Puppeteer.

Community
Advanced
mupengi-botmupengi-bot
11

data-scraper

Fetch HTML from URLs and extract structured data into JSON or Markdown.

Community
Intermediate
davpatel605-beepdavpatel605-beep

scrapling

Extracts web content from static and dynamic websites via HTTP, browser, and stealth methods.

Community
Advanced
tedtv1007-ctrltedtv1007-ctrl

cloudflare-crawl

Fetch rendered HTML from dynamic websites via the Cloudflare Browser Rendering API.

Community
Intermediate