Evaluate completed experiment results and update linked idea status and knowledge graph edges.
npx skills add https://github.com/skyllwt/AutoSci --skill exp-eval-skyllwt
Please help me install this Agent Skill. Skill: exp-eval Source: https://github.com/skyllwt/AutoSci/tree/main/.claude/skills/exp-eval Command: npx skills add https://github.com/skyllwt/AutoSci --skill exp-eval-skyllwt
It turns completed experiment results into an evidence-based verdict that updates the linked idea’s lifecycle status, failure reason, and the graph edges that connect experiments to ideas.
wiki/ideas/{linked-idea}.md fields like status, failure_reason, and date_resolved, plus fills the experiment page’s ## Idea updates section.supports or invalidates edges in wiki/graph/edges.jsonl and rebuilds derived context artifacts (context_brief.md, open_questions.md, and log.md).Run exp-eval on a completed experiment by providing its slug as /exp-eval <experiment-slug> --auto to automatically apply the wiki updates.
High-intent search queries and answers about installing and using this skill.
Automating experiment evaluation involves using an LLM to judge completed results and output a verdict that updates the linked idea's status and the experiment-to-idea knowledge graph edges. It reads wiki pages to apply supported, partially supported, or inconclusive outcomes.
LLM judging for hypothesis validation works by having an impartial Review LLM read experiment and sibling wiki pages to evaluate if results supported, partially supported, refuted, or were inconclusive for the idea’s hypothesis. It then writes status fields and failure reasons.
Evidence synthesis for research lifecycle management converts completed experiment results into an evidence-based verdict that updates the linked idea's lifecycle status, failure reason, and date resolved. It automatically fills the idea updates section on the experiment page.
To add supports or invalidates edges to a knowledge graph, the evaluation process reads experiment results and writes the corresponding edges into the graph edges file. It then rebuilds derived context artifacts like context briefs and open questions.
Yes, you need existing wiki pages for the experiment, linked idea, sibling experiments, and review guidelines before running evaluation. The process requires reading these pages to accurately judge results and update the knowledge graph structure.
Limitations of using wiki automation for experiment evaluation include its dependency on pre-existing, properly linked wiki pages for experiments and ideas. The automated LLM judging cannot evaluate outcomes if the prior research pipeline structure is incomplete.
duany049Evaluate completed experiment metrics and update linked idea statuses.
juicesharpOrchestrate targeted analysis agents to answer structured research questions and generate a consolidated research document.
skyllwtDesign end-to-end experiment suites with ablation-driven iterative refinement for wiki-based research workflows.
Zoomma1Generate and curate a project knowledge graph with graphify.