conversation-evaluator

Evaluate virtual agent conversations for quality by scoring coherence, accuracy, slot filling, intent matching, and hallucination detection. The skill audits conversational AI deployments to identify weaknesses and generate per-dimension quality scorecards.

Yes, you can audit ServiceNow virtual agent topics and agent interactions across multiple deployment channels. The skill relies on MCP and REST data access to fetch conversations, messages, topic definitions, and actions for comprehensive evaluation.

Score conversational AI accuracy and slot filling by validating mandatory input capture and verifying responses align with ServiceNow data and knowledge bases. The skill generates aggregate quality scores and per-dimension metrics for auditing topic design.

Coherence analysis assesses logical flow and context retention across conversation turns. The skill evaluates whether the virtual agent maintains conversational continuity, generating structured evaluation dimensions that highlight where topic design improvements are needed.

Yes, MCP and REST data access are required to fetch conversations, messages, topic definitions, and actions. The skill also requires a SKILL.md frontmatter with a name and description to function properly during enterprise conversational AI audits.

Topic definitions and actions data are required because the skill verifies accuracy against ServiceNow data and validates slot filling against intended topic logic. Without these inputs, hallucination detection and intent matching evaluations cannot produce reliable quality scorecards.