check-test-logical-resp

You can audit passed test results for semantic sanity by scanning test-result JSON files to verify logical coherence between prompts and model responses. It detects topic, entity, and intent mismatches that surface-level checks miss.

Illogical responses detected in a test-results JSON file include topic, entity, and intent inconsistencies. The audit identifies semantic mismatches where responses appear correct but lack logical coherence with the original prompt.

Yes, you can check logical coherence across multi-model test outputs. The audit analyzes both single-run and multi-model testResults structures, aggregating data to surface any illogical responses with a concise rationale.

Scan test-results JSON by targeting the testResults and modelResults arrays. The audit isolates passed tests, verifies semantic consistency against the prompt, and generates a concise report flagging off-topic model responses.

The best way to verify intent consistency is auditing passed test results by parsing test-result JSON structures. This approach validates logical coherence across test cases, reporting intent mismatches with actionable rationale.