skeptic

Verify or refute agent claims with evidence and a confidence score.

Updated May 9, 2020

Search Tags:#confidence scoring #claim verification #evidence gathering #agent claims #adversarial evaluation

Ready to run

npx skills add https://github.com/tkstang/sequence --skill skeptic-tkstang

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: skeptic
Source: https://github.com/tkstang/sequence/tree/main/.agents/skills/skeptic
Command: npx skills add https://github.com/tkstang/sequence --skill skeptic-tkstang

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps users verify or refute claims made by agents by gathering evidence from various sources.

Core Features & Use Cases

Claim Verification: Adversarially gathers evidence to verify or refute claims.
Adversarial Evaluation: Finds contradicting evidence first, then supporting evidence.
Confidence Score: Returns an inline verdict with a confidence score.
Use Case: When a user questions an agent's claim, the Skeptic Skill can be used to verify the claim with evidence.

Quick Start

Use the /skeptic command to verify a claim made by an agent.

Frequently Asked Questions about skeptic

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I verify claims made by an agent with evidence?▼

To verify agent claims, you can use the /skeptic command to adversarially gather evidence and return a verdict with a confidence score. It searches for contradicting evidence first, then supporting evidence.

What is adversarial evaluation for claim verification?▼

Adversarial evaluation is a claim verification method that actively seeks contradicting evidence before looking for supporting evidence. This approach ensures a rigorous confidence score by stress-testing the agent's initial statement.

Can I use WebSearch and Bash to gather evidence for claim verification?▼

Yes, claim verification can utilize tools like WebSearch, WebFetch, Bash, Read, Glob, and Grep to gather evidence. These tools allow the system to access files and the web to validate agent statements.

Does claim verification require a specific environment to work?▼

Yes, adversarial claim verification requires access to a Claude Code environment to function. It relies on this environment to execute tools like Bash, WebFetch, and Agent for evidence gathering.

How do I get a confidence score for an agent's statement?▼

You can get a confidence score for an agent's statement by running the /skeptic command. It evaluates the claim by gathering evidence and returns an inline verdict alongside the score.

What is the best way to refute an agent claim that seems incorrect?▼

The best way to refute an agent claim is through adversarial evaluation, which prioritizes finding contradicting evidence from various sources. It provides a clear verdict and confidence score based on the gathered facts.