workflow-metrics

To track multi-agent AI workflows, you log agent trajectories and tool usage in a structured format using a unique ID. This Skill captures multi-step agent workflows and applies decorator-based automatic tracking to record the full execution path.

Yes, you can evaluate multi-agent workflows using Azure Foundry. The Skill computes 13 cloud-based evaluation metrics via Azure Foundry and 1 local metric, requiring the AZURE_AI_PROJECT_ENDPOINT to enable cloud evaluations.

You compute plan adherence and evaluation metrics by logging a plan, recording tool steps, and finalizing the workflow trajectory. This triggers local and cloud metric computation, generating 14 evaluation metrics for multi-agent AI workflows.

Azure OpenAI is optional for generating tool definitions. You need AZURE_AI_PROJECT_ENDPOINT for cloud evaluations, while AZURE_OPENAI_JUDGE is optional. Tool definitions can be generated from functions or JSON for Azure OpenAI integration.

To analyze AI workflow metrics, you need the azure-ai-projects, azure-identity, python-dotenv, openai, and azure-ai-evaluation Python dependencies. These enable trajectory logging, cloud evaluations, and tool-definition generation.

Yes, this Skill supports local evaluators alongside cloud-based metrics. It provides 13 cloud-based evaluation metrics via Azure Foundry and 1 local evaluator, allowing combined local and cloud metric computation for multi-agent workflows.