skills.rest
loteiron
loteironCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

blocked-page-recovery

Recover blocked or paywalled web pages from Wayback Machine, archive.today, and Jina Reader.

Updated Sep 10, 2026
Search Tags:#caching#web scraping#paywall bypass#waf#wayback machine#page recovery#archive.today
One-click install
npx skills add https://github.com/loteiron/ZeusAgent --skill blocked-page-recovery-loteiron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: blocked-page-recovery
Source: https://github.com/loteiron/ZeusAgent/tree/main/skills/web/blocked-page-recovery
Command: npx skills add https://github.com/loteiron/ZeusAgent --skill blocked-page-recovery-loteiron

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Web fetches frequently fail with 403/429 errors, paywalls, Cloudflare interstitials, or bot-detection walls, leaving you without the content you need. This Skill recovers copies of blocked pages from third-party archive and rendering services instead of giving up or looping on the same URL. ## Core Features & Use Cases - Ordered recovery ladder: Tries the Wayback Machine first, then archive.today domain rotation (archive.ph, .md, .li, .is), then Jina Reader when JINA_API_KEY is set, with guidance for API-first pivots and browser fallback. - Fake-success detection: Validates every candidate body with byte floors, interstitial-title rejection, and redirect-stub detection so dead Google Cache pages, AMP stubs, and rate-limit HTML are never mistaken for real content. - Provenance discipline: Labels every recovery as snapshot or live and enforces citing archived copies with their snapshot date. - Use Case: A news article returns a paywall and Cloudflare challenge. Run the bundled script to pull the archived copy from archive.today, then cite it as a dated snapshot rather than the live page. ## Quick Start Use the blocked-page-recovery skill to fetch the content of this blocked URL and tell me whether the result is a live copy or an archived snapshot.

Frequently Asked Questions about blocked-page-recovery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I recover a webpage blocked by a paywall or 403 error?▼

Run the bundled recover_page.py script with the blocked URL. It tries the Wayback Machine first, then archive.today mirrors, then Jina Reader, returning the first validated copy with its provenance and source URL.

What is the best way to bypass Cloudflare bot detection for fetching pages?▼

Instead of fighting the WAF, fetch a third-party copy: Wayback Machine snapshots, archive.today user-submitted archives, or Jina Reader's server-side render. The script tries these in order and validates each response body.

Does the Jina Reader route work without an API key?▼

No. Anonymous access to r.jina.ai returns 401 with a Turnstile challenge, so the script skips the Jina route entirely when JINA_API_KEY is unset. Set the environment variable to enable live server-side rendering.

Why does Google Cache not work for recovering pages anymore?▼

Google Cache was discontinued in mid-2024. Requests to webcache.googleusercontent.com return HTTP 200 with a Google Search interstitial and JS redirect, not cached content, so the script rejects it as a fake success.

Can I cite an archived snapshot as current page content?▼

No. Snapshots from Wayback or archive.today may be stale and must be cited with their snapshot date. For current data like prices or breaking news, treat a snapshot as context and state its age explicitly.

What should I try when no archive copy of a URL exists?▼

Pivot to the site's data endpoints: look for /api/, /graphql, .json variants, RSS feeds, or sitemap.xml entries that WAFs protect less aggressively. As a last resort, use a real browser tool to render the page.

Related Skills

Explore More Education & Research Skills→
danbridanbri
1

fetch-sitemap

Fetches and parses XML sitemaps and RSS/Atom feeds to inventory UK Parliament web pages.

Community
Intermediate
NousResearchNousResearch
239.1k

blocked-page-recovery

Recover blocked, paywalled, or WAF-protected web pages from third-party archives.

Official
Advanced
shimo4228shimo4228

cross-source-fact-verification

Cross-validate factual claims in drafts against multiple independent sources.

Community
Intermediate
NousResearchNousResearch
239.1k

scrapling

Scrape websites with stealth browsing and Cloudflare bypass.

Official
Intermediate