skills.rest
TylerSimons1127
TylerSimons1127COMMUNITY
Explore all skills by this author →
GitHub

blocked-page-recovery

Recover blocked or paywalled web pages via Wayback, archive.today, and Jina Reader fallbacks.

Updated Aug 21, 2026
Search Tags:#web scraping#paywall bypass#waf#web archiving#wayback machine#page recovery#archive.today
One-click install
npx skills add https://github.com/TylerSimons1127/vibe --skill blocked-page-recovery-tylersimons1127
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: blocked-page-recovery
Source: https://github.com/TylerSimons1127/vibe/tree/main/skills/research/blocked-page-recovery
Command: npx skills add https://github.com/TylerSimons1127/vibe --skill blocked-page-recovery-tylersimons1127

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Web pages often fail to fetch due to 403/429 errors, Cloudflare interstitials, paywalls, or bot detection, leaving you without the content you need. This Skill works down a ladder of third-party archive and rendering services to retrieve a validated copy instead of giving up or retry-looping the same URL. ## Core Features & Use Cases - Fallback Ladder: Tries Wayback Machine, archive.today domain rotation, and Jina Reader (when JINA_API_KEY is set) in cheapest-first order, with an API-first pivot and real browser as last resorts. - Fake-Success Detection: Validates every candidate body using byte floors, interstitial-title rejection, and redirect-stub detection so dead Google Cache pages, AMP stubs, and 429 rate-limit bodies are never mistaken for real content. - Provenance Discipline: Labels every recovery as snapshot or live so citations can include archive dates and stale snapshots are never presented as current data. - Use Case: A news article returns a Cloudflare "Just a moment..." page. Run the bundled script to pull the Wayback snapshot with its timestamp and cite it as archived on that date. ## Quick Start Ask the agent to recover the blocked article at a given URL using the blocked-page-recovery script and report which route succeeded with its provenance.

Frequently Asked Questions about blocked-page-recovery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I recover a webpage blocked by Cloudflare or a paywall?▼

Run the recover_page.py script with the blocked URL. It tries the Wayback Machine first, then archive.today domain rotation, then Jina Reader if JINA_API_KEY is set, returning the first validated copy with its provenance.

What is the best way to access an archived version of a deleted page?▼

Query the Wayback Machine available API at archive.org/wayback/available with the target URL to get the closest snapshot URL and timestamp. For enumerating many snapshots, use the CDX search index, falling back to the available API if CDX returns 503.

Does Jina Reader work without an API key?▼

No, anonymous access to r.jina.ai is dead and returns 401 with a Turnstile challenge. The Jina route is skipped entirely when the JINA_API_KEY environment variable is unset, so set the key before relying on it.

Why does Google Cache not work for recovering pages anymore?▼

Google Cache was discontinued in mid-2024. webcache.googleusercontent.com now returns HTTP 200 with a Google Search interstitial and JS redirect rather than a cached copy, so it must never be treated as a successful recovery.

How can I tell if an archive.today fetch actually succeeded?▼

Validate the body, not the status code, since archive.today returns multi-KB rate-limit HTML even on 429 responses. Check for the target page's actual content such as title words or expected strings rather than response size alone.

When should I use a real browser instead of archive services?▼

Use the browser tool only as the last resort after archives and API pivots fail, since it is the most expensive route. Archives also fail for robots-blocked sites, never-crawled URLs, and JS-only SPAs that snapshots cannot render.

Related Skills

Explore More Education & Research Skills→
unclejobs-aiunclejobs-ai
37

unblock

Retrieve validated page content from blocked URLs via adaptive fetch escalation.

Community
Advanced
krzemienskikrzemienski
2

documentation-research

Fetch and cite current upstream documentation for external dependencies.

Community
Advanced
hanmujunhanmujun
1

graphql-methodology

Tests GraphQL endpoints for introspection leaks, injection, and authorization bypasses.

Community
Intermediate
IceWhaleTechIceWhaleTech
4

browser

Operate a real browser session to interact with JavaScript-heavy pages.

Official
Intermediate