Diagnose stalled AI evaluations in the Hawk/Inspect framework.
npx skills add https://github.com/METR/inspect-action --skill debug-stuck-eval
Please help me install this Agent Skill. Skill: debug-stuck-eval Source: https://github.com/METR/inspect-action/tree/main/.claude/skills/debug-stuck-eval Command: npx skills add https://github.com/METR/inspect-action --skill debug-stuck-eval
This Skill helps diagnose and resolve issues when AI evaluations are not progressing as expected, preventing wasted time and resources.
Use the debug-stuck-eval skill to check the status of eval-set-id '12345'.
High-intent search queries and answers about installing and using this skill.
To fix a stuck eval in Hawk, check the evaluation status with `hawk status`, analyze `hawk logs` for retry loops or 500 errors, and use `curl` to test the API connection directly to isolate the root cause.
An Inspect AI evaluation may hang due to API connectivity issues or server errors. Reviewing logs for repeated retry requests or 500 errors helps identify whether the runner is stuck waiting for unresponsive external endpoints.
The best way to troubleshoot 500 errors in an AI evaluation is to use direct API testing via `curl` alongside log analysis. This isolates whether the failure originates from the evaluation runner or the external API provider.
You can check the status of a frozen eval set by running the `hawk status` command with the specific eval-set-id. This quickly assesses the state of the running evaluation to determine if it has stalled.
To break a retry loop causing an eval timeout, use `hawk logs` to pinpoint the failing requests, then test the targeted endpoint with `curl` to verify API health and resolve the underlying connection issue.
METRList, inspect, and retrieve detailed results from Hawk AI evaluations.
profmikegreeneAutomate ETL pipelines for pilot analytics projects.
superfhpInteract with the SiYuan Notebase API using curl commands.
ckorhonenDebug Next.js server-side errors from terminal and server logs.
