skills.rest
vikingokft
vikingokftOFFICIAL
Explore all skills by this author →
GitHub

web-scraping

Automate web data extraction using traffic interception, sitemaps, APIs, and DOM scraping.

Updated Jun 7, 2026
Search Tags:#web-scraping#APIFY#crawlee#adaptive-strategy#API-discovery#sitemap-discovery
One-click install
npx skills add https://github.com/vikingokft/vikingo-studio-skills --skill web-scraping-vikingokft
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-scraping
Source: https://github.com/vikingokft/vikingo-studio-skills/tree/main/web/web-scraper
Command: npx skills add https://github.com/vikingokft/vikingo-studio-skills --skill web-scraping-vikingokft

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates the complex workflow of extracting structured data from varied websites by coordinating traffic interception, sitemap-based URL discovery, API data fetches, and DOM scraping into cohesive, reusable workflows.

Core Features & Use Cases

  • Adaptive strategy selection across traffic interception, sitemap-based URL discovery, API data fetches, and DOM scraping to maximize data quality and speed.
  • Production-ready guidance for TypeScript Actors via Apify CLI, with modular, progressive-disclosure organization.
  • Real-world workflows for e-commerce, news, and content sites that require robust extraction, validation, and self-critique reporting.

Quick Start

Run this skill against a target site to automatically select and execute the most effective extraction strategy.

Frequently Asked Questions about web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is the best way to extract structured data from websites that use anti-bot measures?▼

Web scraping orchestration bypasses anti-bot measures by dynamically selecting among traffic interception, sitemap discovery, API data retrieval, and DOM scraping, enforcing validation and safe fallbacks to ensure reliable structured data extraction.

How do I choose between sitemap discovery and traffic interception for web scraping?▼

Adaptive strategy selection automatically evaluates the target site to choose between sitemap-based URL discovery, traffic interception, API data retrieval, or DOM scraping, maximizing data quality and extraction speed without manual configuration.

Does this web scraping approach support TypeScript Actors via the Apify CLI?▼

Yes, the web scraping orchestration provides production-ready guidance for modular TypeScript Actors via the Apify CLI, enabling scalable deployments and progressive-disclosure organization for real-world extraction workflows.

Can I use DOM scraping and API data retrieval together for e-commerce data extraction?▼

Yes, hybrid approaches combine DOM scraping and API data retrieval to extract e-commerce data, coordinating multiple extraction strategies into a cohesive workflow with validation and self-critique reporting to handle complex site structures.

Why does my web scraper fail on dynamic content sites, and how can I fix it?▼

Web scrapers fail on dynamic content due to changing DOM structures and anti-bot blocks; adaptive orchestration fixes this by falling back to traffic interception or API data retrieval and enforcing validation and robust error handling.

Related Skills

Explore More Software Engineering Skills→
rawvegrawveg
40

apify-js-sdk

Build and operate Apify Actors and web scrapers using the JavaScript SDK.

Community
Advanced
shynlee04shynlee04

web-search

Search the web and extract content via Tavily and Exa using the inference.sh CLI.

Community
Intermediate
Gull-StackGull-Stack

social-media

Automate social media strategy, content planning, and engagement optimization across platforms.

Official
Basic
lisposterlisposter
1

baoyu-post-to-x

Automate X posts, media uploads, and articles via Chrome DevTools Protocol.

Community
Advanced