skills.rest
Lingjie-wang
Lingjie-wangCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

auto-review-loop

Automates multi-round adversarial research review cycles with external reviewer backends until acceptance criteria are met.

Updated Jul 9, 2026
Search Tags:#codex#iterative improvement#peer review#state persistence#research workflow#ml experiments#autonomous review
One-click install
npx skills add https://github.com/Lingjie-wang/autoRL --skill auto-review-loop-lingjie-wang
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: auto-review-loop
Source: https://github.com/Lingjie-wang/autoRL/tree/main/Auto-claude-code-research-in-sleep/skills/auto-review-loop
Command: npx skills add https://github.com/Lingjie-wang/autoRL --skill auto-review-loop-lingjie-wang

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Iterating on ML research work against critical feedback is slow and manual: you send drafts to a reviewer, wait, implement fixes, and repeat. This Skill automates the entire review-fix-re-review loop, driving an external reviewer (Codex or manual backend) through up to 4 rounds until the work scores 6/10 or higher with a "ready" or "almost" verdict. ## Core Features & Use Cases - Autonomous review loop: Repeatedly sends artifacts to an external reviewer, parses score/verdict/weaknesses, implements fixes, runs experiments, and re-reviews — with state persisted to REVIEW_STATE.json so it survives context compaction. - Three difficulty levels: medium (MCP-based review), hard (adds reviewer memory and a debate/rebuttal protocol), and nightmare (reviewer reads the repo directly via codex exec, verifying claims against code). - Full audit trail: Saves raw reviewer responses verbatim, debate transcripts, and review traces to review-stage/AUTO_REVIEW.md, with optional HTML rendering and Feishu notifications. - Use Case: You have a NeurIPS submission draft with experiment results. Run the loop overnight; it adversarially reviews the paper, fixes flagged weaknesses, reruns experiments, and stops when the reviewer judges it submission-ready. ## Quick Start Run the auto review loop on my current paper draft and experiment results, reviewing until it passes or reaches 4 rounds.

Frequently Asked Questions about auto-review-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run an automated review loop on my research paper?▼

Invoke the skill with a topic or scope, and it sends your paper draft, code, and raw results to an external reviewer, implements the flagged fixes, and re-reviews. The loop stops when the score reaches 6/10 with a ready or almost verdict, or after 4 rounds.

What reviewer backends does the auto review loop support?▼

It supports a Codex backend via MCP tools, a manual review backend via manual-review MCP, and codex exec for nightmare difficulty where the reviewer reads the repository directly. The default backend is Codex with xhigh reasoning effort.

What is the difference between medium, hard, and nightmare review difficulty?▼

Medium uses standard MCP-based review where the executor controls reviewer context. Hard adds persistent reviewer memory and a debate protocol where the executor can rebut criticisms. Nightmare lets the reviewer read the repo directly via codex exec and independently verify claims against code.

Can the review loop recover after context window compaction?▼

Yes, it writes REVIEW_STATE.json after every round with the round number, threadId, score, and pending experiments. On restart, states marked in_progress within 24 hours resume from the next round; completed or stale states trigger a fresh start.

When does the auto review loop stop iterating?▼

The loop stops when the reviewer gives a score of at least 6/10 AND a verdict of ready or almost — both conditions must hold. It also stops after 4 rounds, listing remaining blockers and effort estimates if no positive assessment was reached.

Is nightmare difficulty compatible with the manual review backend?▼

No, nightmare mode requires Codex CLI and codex exec so the reviewer can read the repository directly, which the manual backend cannot do. Use hard difficulty with manual review, or switch the backend to codex.

Related Skills

Explore More Education & Research Skills→
lightrain-alightrain-a

paper-plan

Generate a structured paper outline from review conclusions and experiment results.

Community
Advanced
wanghao9610wanghao9610
2

stage-peer-reviewer

Simulates a five-perspective peer review panel that scores manuscripts against anchored rubrics.

Community
Advanced
hao-ai-labhao-ai-lab
3.9k

monitor-experiment

Monitor W&B training runs to detect anomalies and generate alerts.

Official
Intermediate
czh-ee-2023czh-ee-2023

auto-review-loop-minimax

Automate multi-round research review loops using the MiniMax API.

Community
Advanced