skills.rest
virahitvin8
virahitvin8COMMUNITY
Explore all skills by this author →
GitHub

baoyu-url-to-markdown

Convert any URL to markdown using Chrome CDP with site-specific extraction adapters.

2|Updated May 30, 2026
Search Tags:#web scraping#content extraction#url-to-markdown#chrome cdp#web archiving#youtube transcript#twitter archive
One-click install
npx skills add https://github.com/virahitvin8/crafty-gis --skill baoyu-url-to-markdown-virahitvin8
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: baoyu-url-to-markdown
Source: https://github.com/virahitvin8/crafty-gis/tree/main/GIT_STAR/assignment-generator/antigravity-skills-main/antigravity-skills-main/skills/baoyu-url-to-markdown
Command: npx skills add https://github.com/virahitvin8/crafty-gis --skill baoyu-url-to-markdown-virahitvin8

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @mozilla/readability, chrome-launcher, defuddle, jsdom, remark-gfm, remark-parse, turndown, turndown-plugin-gfm, unified, ws, and includes scripts (resource) and references (resource) components.

What problem does it solve? Saving web pages as clean markdown often fails because of login walls, CAPTCHAs, JavaScript-heavy rendering, and cluttered page layouts. This Skill fetches any URL through a real Chrome browser via the DevTools Protocol and converts the content into structured markdown, handling authentication gates and noisy pages automatically. ## Core Features & Use Cases - Site-Specific Adapters: Built-in extractors for X/Twitter (tweets, threads, articles), YouTube (transcripts with timestamps), Hacker News (threaded comments), and a generic Defuddle/Readability fallback for any other page. - Login & CAPTCHA Handling: Interaction wait modes open a visible Chrome window, auto-detect login or challenge gates, and resume extraction once cleared. - Media Download & Quality Gate: Optionally downloads images and videos to local folders with rewritten markdown links, plus a quality-check workflow to detect low-quality headless captures. - Use Case: Save a long X thread or a YouTube video transcript as a local markdown file with all images downloaded, organized by domain and slug for a personal knowledge base. ## Quick Start Ask the agent to save a webpage as markdown, for example: fetch https://example.com/article and save it as markdown with images downloaded.

Frequently Asked Questions about baoyu-url-to-markdown

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a webpage to markdown from the command line?▼

Run baoyu-fetch followed by the URL, optionally adding --output to save to a file. The CLI renders the page in Chrome via CDP and extracts clean markdown using site adapters or a Defuddle/Readability fallback.

How to save a YouTube video transcript as markdown?▼

Pass the YouTube URL to baoyu-fetch and the youtube adapter auto-activates, extracting captions as timestamped transcript segments with chapter headings. Videos without exposed caption tracks produce description-only output.

Can I fetch pages that require login or have CAPTCHA protection?▼

Yes, use --wait-for interaction to open a visible Chrome window that auto-detects login forms, Cloudflare challenges, reCAPTCHA, or hCaptcha and continues once cleared. Use --wait-for force for manual control with Enter to continue.

Does the tool download images and videos from saved pages?▼

Yes, the --download-media flag saves images to an imgs/ folder and videos to a videos/ folder next to the output file, rewriting markdown links to local relative paths. It requires --output to be set.

Why does headless capture sometimes return low-quality markdown?▼

Some sites render differently in headless Chrome, returning login shells, error pages, or framework payloads without failing the CLI. Inspect the output after each run and retry with --wait-for interaction or --wait-for force when content looks wrong.

What runtime dependencies does baoyu-fetch need?▼

The CLI requires the Bun runtime and a Chrome or Chromium browser. Third-party npm packages like defuddle, jsdom, turndown, and chrome-launcher are installed via bun install in the scripts directory.

Related Skills

Explore More Content & Communication Skills→
buildinternetbuildinternet
3

finding-changelogs

Evaluate changelog URLs and recommend optimal ingestion methods.

Official
Advanced
alchaincyfalchaincyf
1.5k

huashu-prompt-save

Classifies prompts into five categories and saves them as indexed Markdown files.

Community
Basic
Executioner1939Executioner1939
8

sort-unsorted

Re-classify unsorted INDEX rows using repository metadata and taxonomy rules.

Community
Intermediate
lodekeeperlodekeeper
4

web-scraping

Scrape web content with tiered HTTP and headless browser automation.

Community
Advanced