skills.rest
mrvarmazyar
mrvarmazyarCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-research

Search and fetch web pages into clean markdown with sources.

8|2|Updated May 8, 2026
Search Tags:#summarization#caching#web research#Groq#MCP server#documentation lookup#TinyFish
One-click install
npx skills add https://github.com/mrvarmazyar/web-research --skill web-research-mrvarmazyar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-research
Source: https://github.com/mrvarmazyar/web-research/tree/main
Command: npx skills add https://github.com/mrvarmazyar/web-research --skill web-research-mrvarmazyar

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Web research for AI agents often becomes too expensive and noisy when raw HTML and irrelevant page clutter are sent into the model context.

Core Features & Use Cases

  • Clean, token-efficient web context with sources: Searches, fetches, converts HTML to markdown, and returns focused content instead of dumping scripts, navbars, ads, and cookie banners.
  • Unified CLI + MCP tools: Use the wr CLI for search, fetch, and research, or use the wr-mcp server to expose tools like web_search, web_fetch, and web_research.
  • Multi-mode retrieval: Supports summarize (LLM), lossless (full cleaned markdown), and chunks (TF-IDF relevant sections) with an on-disk cache for speed.

Quick Start

Run wr research with your question: wr research "stripe webhook idempotency next.js app router".

Frequently Asked Questions about web-research

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fetch and summarize web pages for AI agents without exceeding token limits?▼

Web research for AI agents returns token-efficient markdown with sources by stripping noisy HTML, scripts, and ads. It fetches and summarizes public pages specifically to reduce content for model context.

What is the best way to retrieve clean markdown from web pages for documentation lookup?▼

Documentation lookup is handled by converting raw HTML into focused markdown. The pipeline performs parallel search and concurrent fetch with optional Jina fallback, returning clean text instead of page clutter.

Can I use an MCP server to expose web search and fetch tools to my AI agent?▼

Yes, you can use the wr-mcp server to expose web_search, web_fetch, and web_research tools. This allows AI agents to perform web research directly through the MCP protocol.

Does this web research pipeline support caching and different summarization modes?▼

The pipeline provides summarize, lossless, and chunks retrieval modes. It uses an on-disk cache with an expiration TTL to speed up repeated web search and fetch requests.

How do I run web research queries from the command line?▼

You run web research queries from the command line using the wr CLI. Execute a command like wr research followed by your question to fetch and summarize web content.

Why does sending raw HTML to my AI model context become too expensive?▼

Raw HTML contains scripts, navbars, ads, and cookie banners that waste tokens. Web research solves this by fetching, cleaning, and summarizing pages into focused markdown before sending it to the model.

Related Skills

Explore More Education & Research Skills→
FISHers6FISHers6
32

setup

Configure ClawBro gateway, API keys, and operating modes via config files.

Community
Intermediate
buiphucminhtambuiphucminhtam
47

memory-manager

Store project decisions and progress as git-committed JSONL records.

Community
Advanced
githubgithub
37.6k

mcp-cli

Execute MCP server tools with JSON payloads via CLI.

Official
Basic
JKHeadleyJKHeadley
75

smart-web-fetch

Fetch clean readable web content from URLs via llms.txt and markdown fallback.

Community
Intermediate