Agent Evaluation

To evaluate AI agent performance, this skill applies a structured scoring rubric across multiple dimensions including architecture, security, operations, testing, and documentation to objectively measure and score agent capabilities.

Tracking AI agent performance trends is achieved by comparing current evaluation scores against historical baseline scores. This comparison highlights performance shifts and generates actionable improvement recommendations for the agent.

Assessing an AI coding assistant involves evaluating its outputs against predefined quality dimensions like security and architecture. The skill analyzes agent configurations and invocation history to generate a detailed evaluation report.

Generating actionable improvement recommendations requires read access to agent configurations, output logs, and telemetry data. The skill uses this data to evaluate operations and testing dimensions, writing final assessment reports.

Evaluating an agent without telemetry data is not recommended. The skill requires read access to agent configurations, output logs, and telemetry to accurately score operations, testing, and documentation dimensions for a complete evaluation.

Automated agent evaluation supports security specialist agents by scoring their performance across security, architecture, and operations dimensions. It analyzes invocation history and telemetry to ensure the agent meets predefined quality standards.