novelty-check

Verifies research idea novelty against recent literature using multi-source search and external LLM review.

75|7|Updated May 2, 2026
One-click install
npx skills add https://github.com/zjunlp/Mechanist --skill novelty-check-zjunlp
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: novelty-check
Source: https://github.com/zjunlp/Mechanist/tree/main/skills/novelty-check
Command: npx skills add https://github.com/zjunlp/Mechanist --skill novelty-check-zjunlp

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Researchers risk wasting months implementing methods that already exist in the literature. This Skill systematically checks whether a proposed method or idea is novel before committing to implementation, comparing it against recent papers from arXiv, major ML venues, and web sources. ## Core Features & Use Cases - Claim Extraction: Breaks a method description into 3-5 core technical claims that each need independent novelty verification. - Multi-Source Literature Search: Searches arXiv, Google Scholar, Semantic Scholar, and recent conference proceedings (ICLR, NeurIPS, ICML) with multiple query formulations per claim. - Cross-Model Verification: Sends findings to an external LLM reviewer via the llm-chat MCP for strict, high-rigor feedback on closest prior work. - Structured Novelty Report: Produces a scored report with per-claim novelty ratings, closest prior work table, an overall PROCEED / PROCEED WITH CAUTION / ABANDON recommendation, and positioning suggestions. - Use Case: Before implementing a new mechanistic interpretability probing method, run a novelty check to discover a nearly identical 2025 arXiv preprint and reposition the contribution around a different experimental setting. ## Quick Start Ask the agent to run a novelty check on your proposed method by describing the idea, for example: "Check the novelty of using sparse autoencoders to steer attention heads in small language models."

Frequently Asked Questions about novelty-check

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if my research idea is novel?

Provide a description of your proposed method and the Skill extracts 3-5 core claims, searches arXiv and recent conference proceedings for each claim, then consults an external LLM reviewer. It outputs a scored report with a PROCEED, PROCEED WITH CAUTION, or ABANDON recommendation.

What literature sources does a novelty check cover?

The search covers arXiv, Google Scholar, and Semantic Scholar via web search, plus recent proceedings from ICLR 2025/2026, NeurIPS 2025, and ICML 2025/2026. It emphasizes the most recent six months of arXiv preprints since the field moves quickly.

What counts as a novel research contribution?

An idea is novel if it proposes a new method, theory, or task, applies an existing method to an unstudied domain, gives a new explanation or mechanism for an existing problem, or uses a new experimental setting or dataset. Applying a method within the same domain is not novel without surprising insights.

Can a novelty check distinguish method novelty from finding novelty?

Yes. The report explicitly states when the method itself is not novel but the resulting finding would be. Each core claim receives its own HIGH, MEDIUM, or LOW novelty rating with the closest overlapping paper identified.

What are the limitations of automated novelty checking?

The check depends on what is indexed and retrievable via web search, so very recent or obscure unpublished work may be missed. It also relies on the external LLM reviewer's judgment, so borderline cases still benefit from human expert review.