extract

Extracts WebLogic JMS messages from a LIF file and republishes them to a JMS topic or queue.

3|1|Updated Jun 14, 2021

Search Tags:#automation #markdown #api #content #web #extract #tavily

Ready to run

npx skills add https://github.com/xdanger/dotfiles --skill extract-xdanger

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: extract
Source: https://github.com/xdanger/dotfiles/tree/main/agents/skills/extract
Command: npx skills add https://github.com/xdanger/dotfiles --skill extract-xdanger

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires jq, curl, node, and includes scripts (resource) components.

What problem does it solve?

Automates extraction of readable content from web pages by calling Tavily's extraction API and returning clean markdown or text, simplifying content gathering for research, documentation, and knowledge sharing.

Core Features & Use Cases

Authentication via Tavily MCP (OAuth flow) or API key for seamless access.
Batch extraction: process multiple URLs with optional query, chunks_per_source, and extract_depth for JS-heavy pages.
Flexible output: choose markdown or plain text formats and optionally include images.
Use Case: convert a set of articles into shareable notes, summaries, or knowledge base content.

Quick Start

Run the extract.sh script with a JSON payload describing the URLs to process.

Frequently Asked Questions about extract

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract clean markdown content from web pages automatically?▼

To extract clean markdown content from web pages automatically, you can run a script that calls Tavily's extraction API to fetch URLs and return readable text. This simplifies gathering content for research or documentation without manual copying.

Can I batch process multiple URLs for content extraction?▼

Yes, you can batch process multiple URLs for content extraction by passing a JSON payload to the processing script. This allows you to convert a set of articles into shareable notes or knowledge base content in a single run.

How do I extract content from JavaScript-heavy web pages?▼

To extract content from JavaScript-heavy web pages, the extraction process supports an extract_depth parameter. Setting this option enables targeted fetching for complex pages, ensuring you retrieve the full rendered markdown or text successfully.

Do I need an API key to use Tavily for web content extraction?▼

You need either an API key or an OAuth authentication flow via Tavily MCP to use Tavily for web content extraction. Once authenticated, the script can communicate with Tavily endpoints to fetch your requested URL results.

What's the best way to convert web articles into plain text for notes?▼

The best way to convert web articles into plain text for notes is using an automated extraction API that supports flexible output formats. You can choose plain text or markdown and optionally include images for your documentation.

Does URL content extraction support query-based reranking?▼

Yes, URL content extraction supports query-based reranking alongside parameters like chunks_per_source. This enables targeted extractions by prioritizing the most relevant information from the fetched web pages for your research.