Scrape websites to extract structured data while handling anti-bot protections.
npx skills add https://github.com/AgentComputerAI/torch --skill scrape-agentcomputerai
Please help me install this Agent Skill. Skill: scrape Source: https://github.com/AgentComputerAI/torch/tree/main/skills/scrape Command: npx skills add https://github.com/AgentComputerAI/torch --skill scrape-agentcomputerai
The scrape skill automates first-pass reconnaissance and extraction from websites, turning a URL into a small validated dataset while navigating framework quirks and anti-bot protections so humans don't have to hand-craft scrapers for every site.
Ask the scrape skill to recon and extract a small sample from the provided URL, produce ./output/<slug>.json, and write a reusable skills/sites/<slug>/SKILL.md playbook.
High-intent search queries and answers about installing and using this skill.
To scrape websites with anti-bot protections like Cloudflare or Turnstile, this Skill applies layered escalation using real Chrome debug profiles, stealth Puppeteer, cookie persistence, and proxies to bypass 403s and client-side blocks.
The best way to extract data from client-rendered pages is browser-based scraping via Puppeteer, which handles dynamic content rendering and pagination scenarios to capture structured data and network requests for API replay.
API reverse-engineering during web scraping works by capturing network requests in a browser environment, allowing you to intercept and replay API endpoints directly to extract structured JSON data instead of parsing HTML.
Yes, you can save and reuse a scraping playbook. The Skill writes a reusable site-specific SKILL.md playbook to skills/sites/<slug>/, allowing subsequent runs to skip reconnaissance and execute extraction directly.
No, you do not need to manually monitor long-running tasks. The Skill supports background execution, spawning scrapers as background processes while streaming progress logs to keep you updated on extraction status.
Web scrapers return 403 errors during reconnaissance when anti-bot protections detect automated access. This Skill counters this by escalating from curl assessment to stealth Puppeteer with randomized headers and cookie persistence.
ajianazAutomate Chrome browser control for site exploration and scraping via CDP.
changgengluParse and standardize Excel workbook data across multiple sheets into JSON or CSV.
jesuscottageConfigure Puppeteer with stealth plugins to bypass anti-bot detection.
gyuhaScrape and crawl websites with HTTP requests, browser automation, and anti-bot bypass.
