wiki-research-loop

Auto-grows a wiki via budget-capped BFS research loop over pluggable web, arXiv, and GitHub fetchers.

1|Updated May 6, 2026
One-click install
npx skills add https://github.com/surfingalien/FinSurfing --skill wiki-research-loop-surfingalien
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: wiki-research-loop
Source: https://github.com/surfingalien/FinSurfing/tree/main/.claude/skills/wiki-research-loop
Command: npx skills add https://github.com/surfingalien/FinSurfing --skill wiki-research-loop-surfingalien

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Manually building a knowledge base requires repeated searching, reading, deduplicating, and writing. This Skill automates that cycle: it pops research seeds from a queue, fetches sources, drafts wiki pages, dedupes claims against existing pages, and enqueues follow-up topics until a budget, depth, or convergence limit halts the run. ## Core Features & Use Cases - Autonomous BFS research loop: Pops seeds breadth-first from a SQLite-backed queue, fetches sources via pluggable fetchers (web via DuckDuckGo Lite, arXiv API, GitHub search), compiles pages, and enqueues follow-up seeds up to a max depth. - Cost and safety controls: Halts on USD budget cap, max-pages cap, depth cap, convergence detection (Jaccard novelty below 5% for 3 consecutive pages), a kill-switch file, or privacy guard that blocks non-local fetchers for private wikis. - Auditable output: Each run writes a human-readable Markdown log and a structured JSON stats file under the wiki root, and every page is upserted through the wiki store so FTS5 search stays consistent. - Use Case: Tell the agent to "research memory consolidation in agents" — it seeds the queue, runs the loop overnight, and you wake up to a grown wiki with sourced claims and follow-up pages. ## Quick Start Ask the agent to research a topic by saying "research <topic>" or "grow the <slug> wiki", and it will seed the queue and run the budget-capped loop automatically.

Frequently Asked Questions about wiki-research-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automatically research a topic and build a wiki?

Seed the queue with a query using the seed command, then run the loop with a slug and optional budget, depth, and page caps. The loop fetches sources, compiles pages, and enqueues follow-up seeds until a halt condition trips.

What sources can the research loop fetch from?

Built-in fetchers cover web search via DuckDuckGo Lite, the arXiv API, and GitHub repository search. You can add custom fetchers by dropping a JavaScript module into the fetchers directory following the documented interface.

How do I stop the research loop from spending too much?

Set a budget cap with the --budget-usd flag, the WIKI_LOOP_BUDGET_USD environment variable, or the wiki config. The loop estimates per-fetcher cost before each iteration and halts when the projected total exceeds the cap.

Can I stop a running research loop immediately?

Yes, create a STOP file in the pro-workflow home directory and the loop halts gracefully at the next iteration check. Removing the file allows the next run to resume processing pending seeds.

Does the research loop work with private wikis?

If the wiki config sets private to true, the loop refuses any non-local fetcher and emits a warning. Only manual seed ingestion is allowed, preventing private content from leaking to external APIs.

How does the loop know when a topic is fully researched?

It computes Jaccard token overlap between each new page and the previous three pages. If novelty stays below five percent for three consecutive pages, it declares convergence and halts the run.