skills.rest
bytedance
bytedanceOFFICIAL
Explore all skills by this author →
Website𝕏@ByteDanceOSSGitHub

link-reader

Extract titles and content from URLs into structured JSON.

417|87|Updated Nov 12, 2025
Search Tags:#url#data-extraction#web-content#json-output#api-key#link-reader
One-click install
npx skills add https://github.com/bytedance/agentkit-samples --skill link-reader
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: link-reader
Source: https://github.com/bytedance/agentkit-samples/tree/main/skills/link-reader
Command: npx skills add https://github.com/bytedance/agentkit-samples --skill link-reader

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires volcenginesdkarkruntime, veadk, httpx, and includes scripts (resource) components.

What problem does it solve?

This Skill uses a built-in link_reader function to fetch the title and main content from URLs, enabling quick extraction of structured information from web pages, PDFs, and Douyin videos.

Core Features & Use Cases

  • Read titles and main content from multiple URLs (web pages, PDFs, Douyin videos) and return a structured list of results.
  • Supports batch processing by supplying a URL list and obtaining per-URL data for downstream ingestion, summarization, or QA workflows.
  • Use case: gather summaries for research notes, monitor online content, or build a content ingestion pipeline for an AI agent.

Quick Start

Run the script with your URL list, e.g., python scripts/link_reader.py "https://example.com" "https://example.org".

Frequently Asked Questions about link-reader

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract web content and titles from multiple URLs into structured data?▼

To extract web content from URLs, you can run the link reader script with a list of URL arguments to return a JSON array of objects containing the title and content for each page.

Can I extract content from PDFs and short videos using a URL extraction tool?▼

Yes, URL extraction supports fetching titles and main content from standard web pages, PDFs, and Douyin short videos by passing their links into the URL list.

What is the best way to batch process URLs for a content ingestion pipeline?▼

The best way to batch process URLs for a content ingestion pipeline is to supply a URL list to the async HTTP client, which calls the service endpoint and returns structured JSON data per URL.

Do I need an API key to fetch webpage content programmatically?▼

Yes, you need to configure an API key to authenticate requests when the async HTTP client calls the LinkReader service endpoint to fetch webpage content programmatically.

How does the tool handle errors when fetching data from multiple links?▼

The tool handles errors gracefully during the data extraction process, ensuring that failed requests do not crash the batch operation when fetching content from multiple links.

Why does my URL extraction return a JSON array instead of plain text?▼

Your URL extraction returns a JSON array because the tool is designed to output structured data objects with title and content fields, making it suitable for downstream summarization and AI workflows.

Related Skills

Explore More Data & Analytics Skills→
greenflash-aigreenflash-ai

greenflash-verify

Verify Greenflash integration configuration and data flow for Python and TypeScript projects.

Official
Advanced
alibabaalibaba
10.5k

pdf-extractor

Extract text, tables, and metadata from PDF documents into JSON.

Official
Intermediate
TheBushidoCollectiveTheBushidoCollective
187

rust-async-patterns

Implement async Rust patterns with tokio, async/await, and futures.

Official
Advanced
otrebuotrebu
4

web-to-markdown

Batch-convert JavaScript-rendered web pages to a timestamped Markdown file.

Community
Advanced