skills.rest
SadikinAraf
SadikinArafCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

imperium-crawl — Agent Skill Guide

Convert web data collection goals into deterministic scraping and extraction workflows.

11|Updated Jun 19, 2021
Search Tags:#data extraction#web scraping#crawling#stealth#skill automation#api recon
One-click install
npx skills add https://github.com/SadikinAraf/imperium-crawl --skill imperium-crawl-agent-skill-guide
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: imperium-crawl — Agent Skill Guide
Source: https://github.com/SadikinAraf/imperium-crawl/tree/main/SKILL
Command: npx skills add https://github.com/SadikinAraf/imperium-crawl --skill imperium-crawl-agent-skill-guide

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you go from a vague web data request to reliable, repeatable scraping and extraction workflows without trial-and-error.

Core Features & Use Cases

  • Smart scraping & escalation: Choose readability, scrape, extract, or AI extraction based on page type (static, JS-heavy, blocked).
  • Reusable skill building: Create, run, and verify saved extraction skills for consistent results across similar pages.
  • Site and API intelligence: Map crawls for structure discovery and reverse-engineer hidden endpoints via API recon.
  • Batch and research workflows: Perform bulk scraping with resumable jobs and synthesize multi-source research.

Quick Start

Ask an AI to run the imperium-crawl smart guide to extract structured data from a target URL, then use the provided skill-building flow to create and verify a reusable extraction skill.

Frequently Asked Questions about imperium-crawl — Agent Skill Guide

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from JavaScript-heavy or blocked web pages?▼

To extract structured data from dynamic or blocked pages, this Skill applies smart scraping escalation, automatically choosing between readability, scrape, extract, or AI extraction modes based on the page's complexity.

What is the best way to build reusable web scraping workflows for multiple similar listings?▼

Building reusable web scraping workflows is achieved by creating, running, and verifying saved extraction skills, ensuring consistent data collection results across similar pages without repetitive manual configuration.

How can I discover hidden APIs during site analysis for web scraping?▼

Discovering hidden APIs during site analysis is handled through the Skill's API recon feature, which reverse-engineers underlying endpoints to map crawl structures and extract data directly.

Can I run batch web scraping jobs and resume them if the process stops?▼

Running batch web scraping jobs supports bulk data collection with resumable pipelines, allowing large-scale extraction tasks to pause and resume without losing progress or duplicating requests.

Do I need specific parameters to configure stealth levels for web crawling?▼

Configuring stealth levels for web crawling requires tool-orchestrated usage with correct parameters, including stealth levels, selectors, and schema modes, alongside optional API keys to bypass blocks.

How do I verify that my web scraping extraction skill produces the correct output?▼

Verifying that a web scraping extraction skill produces correct output requires running the built skill through the provided run-skill flow, ensuring the extraction logic matches the expected schema.

Related Skills

Explore More Software Engineering Skills→
s-hiraokus-hiraoku
21

mcp-firecrawl

Scrape websites, crawl pages, search, and extract structured data via Firecrawl tools.

Community
Basic
Wike-CHIWike-CHI
2

document-pro

Extract text, tables, and images from PDFs, DOCX, PPTX, XLSX, and plain text.

Community
Intermediate
openeuler-mirroropeneuler-mirror
3

dev-browser

Automate browser tasks with persistent state using Playwright and Node.js.

Official
Advanced
Prorise-coolProrise-cool
24

scraping-specialist

Scrape websites and extract structured data into CSV or JSON.

Community
Advanced