Evaluate agent run transcripts against a compliance rubric for process fidelity.
npx skills add https://github.com/EdgeApp/edge-dev-agents --skill agent-eval-edgeapp
Please help me install this Agent Skill. Skill: agent-eval Source: https://github.com/EdgeApp/edge-dev-agents/tree/main/.cursor/skills/agent-eval Command: npx skills add https://github.com/EdgeApp/edge-dev-agents --skill agent-eval-edgeapp
This Skill addresses the challenge of verifying autonomous agent behavior by grading completed runs against a strict compliance rubric, ensuring process adherence and outcome honesty.
Run the agent-eval skill to audit the specified agent run by providing its unique Asana GID to the evaluation script.
High-intent search queries and answers about installing and using this skill.
To audit autonomous agent performance, you grade completed run transcripts against a standardized compliance rubric to ensure process fidelity and report honesty. This evaluates post-hoc quality assurance by checking orchestrated development workflows.
Agent run verification involves validating agent claims against live GitHub and Asana state to prevent false completion reports. It requires access to the agent's run manifest, transcript logs, and live repository state to perform evidence-based grading.
Evidence-based grading requires the agent's run manifest, transcript logs, and live repository state. You also need command-line dependencies like gh and jq installed to query repository and task data for compliance auditing.
Yes, agent-eval applies to post-hoc quality assurance of orchestrated development workflows, including PR verification, Asana status transitions, and testing report accuracy. You provide the agent run's unique Asana GID to the evaluation script.
The best way to audit agent runs for workflow deviations is grading them against a comprehensive compliance rubric with dimensions A1-A32. This identifies process deviations and generates a detailed audit report ensuring prescribed workflows were followed.
Compliance auditing requires transcript logs because they provide the evidence needed to evaluate process fidelity and outcome honesty. Without run manifests and logs, the system cannot perform evidence-based grading against the compliance rubric.
EdgeAppResolve agent sessions into structured JSON evidence manifests with transcripts and logs.
cookeyholderCreate a GitHub Pull Request with a Traditional Chinese PR message.
vignesh2027Search local and live GitHub issues and PRs with gitcrawl.
goodnessibehEvaluate AI agent performance using multi-dimensional rubrics for correctness, completeness, and efficiency.
