check-test-logical-resp

Audits passed test results to verify model responses are logically coherent with prompts.

Updated Jan 16, 2026

Search Tags:#QA #test results #semantic sanity #logical coherence #model prompts #test-results JSON

Ready to run

npx skills add https://github.com/dezverev/AnimalAL-v1 --skill check-test-logical-resp

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: check-test-logical-resp
Source: https://github.com/dezverev/AnimalAL-v1/tree/main/.cursor/skills/check-test-logical-resp
Command: npx skills add https://github.com/dezverev/AnimalAL-v1 --skill check-test-logical-resp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill audits passed tests to verify that each model response is logically coherent with the user's prompt, surfacing semantic inconsistencies that pass keyword checks or surface-level tool usage.

Core Features & Use Cases

Semantic sanity auditing of passed test results to detect topic, entity, or intent mismatches.
Aggregates and analyzes test-result JSON to surface illogical or off-topic responses.
Supports both single-run and multi-model test outputs and generates a concise, actionable report.

Quick Start

Run the check on the latest test-results JSON to identify any passed tests with illogical responses. Example: /check-test-logical-resp against the most recent file in src/IntegrationTesterApp/test-results/.

Frequently Asked Questions about check-test-logical-resp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I audit passed test results for semantic sanity and logical coherence?▼

You can audit passed test results for semantic sanity by scanning test-result JSON files to verify logical coherence between prompts and model responses. It detects topic, entity, and intent mismatches that surface-level checks miss.

What illogical responses can be detected in a test-results JSON file?▼

Illogical responses detected in a test-results JSON file include topic, entity, and intent inconsistencies. The audit identifies semantic mismatches where responses appear correct but lack logical coherence with the original prompt.

Can I check logical coherence across multi-model test outputs?▼

Yes, you can check logical coherence across multi-model test outputs. The audit analyzes both single-run and multi-model testResults structures, aggregating data to surface any illogical responses with a concise rationale.

How do I scan test-results JSON to find passed tests with off-topic model responses?▼

Scan test-results JSON by targeting the testResults and modelResults arrays. The audit isolates passed tests, verifies semantic consistency against the prompt, and generates a concise report flagging off-topic model responses.

What's the best way to verify intent consistency in model prompts and test results?▼

The best way to verify intent consistency is auditing passed test results by parsing test-result JSON structures. This approach validates logical coherence across test cases, reporting intent mismatches with actionable rationale.