web-research

Search and fetch web pages into clean markdown with sources.

8|2|Updated May 8, 2026

Search Tags:#summarization #caching #web research #Groq #MCP server #documentation lookup #TinyFish

Ready to run

npx skills add https://github.com/mrvarmazyar/web-research --skill web-research-mrvarmazyar

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: web-research
Source: https://github.com/mrvarmazyar/web-research/tree/main
Command: npx skills add https://github.com/mrvarmazyar/web-research --skill web-research-mrvarmazyar

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web research for AI agents often becomes too expensive and noisy when raw HTML and irrelevant page clutter are sent into the model context.

Core Features & Use Cases

Clean, token-efficient web context with sources: Searches, fetches, converts HTML to markdown, and returns focused content instead of dumping scripts, navbars, ads, and cookie banners.
Unified CLI + MCP tools: Use the wr CLI for search, fetch, and research, or use the wr-mcp server to expose tools like web_search, web_fetch, and web_research.
Multi-mode retrieval: Supports summarize (LLM), lossless (full cleaned markdown), and chunks (TF-IDF relevant sections) with an on-disk cache for speed.

Quick Start

Run wr research with your question: wr research "stripe webhook idempotency next.js app router".

Frequently Asked Questions about web-research

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I fetch and summarize web pages for AI agents without exceeding token limits?▼

Web research for AI agents returns token-efficient markdown with sources by stripping noisy HTML, scripts, and ads. It fetches and summarizes public pages specifically to reduce content for model context.

What is the best way to retrieve clean markdown from web pages for documentation lookup?▼

Documentation lookup is handled by converting raw HTML into focused markdown. The pipeline performs parallel search and concurrent fetch with optional Jina fallback, returning clean text instead of page clutter.

Can I use an MCP server to expose web search and fetch tools to my AI agent?▼

Yes, you can use the wr-mcp server to expose web_search, web_fetch, and web_research tools. This allows AI agents to perform web research directly through the MCP protocol.

Does this web research pipeline support caching and different summarization modes?▼

The pipeline provides summarize, lossless, and chunks retrieval modes. It uses an on-disk cache with an expiration TTL to speed up repeated web search and fetch requests.

How do I run web research queries from the command line?▼

You run web research queries from the command line using the wr CLI. Execute a command like wr research followed by your question to fetch and summarize web content.

Why does sending raw HTML to my AI model context become too expensive?▼

Raw HTML contains scripts, navbars, ads, and cookie banners that waste tokens. Web research solves this by fetching, cleaning, and summarizing pages into focused markdown before sending it to the model.