Extract samples, messages, events, and metrics from Inspect AI .eval logs.
npx skills add https://github.com/tbroadley/dotfiles --skill read-inspect-eval
Please help me install this Agent Skill. Skill: read-inspect-eval Source: https://github.com/tbroadley/dotfiles/tree/main/claude/skills/read-inspect-eval Command: npx skills add https://github.com/tbroadley/dotfiles --skill read-inspect-eval
This Skill helps data analysts and ML engineers quickly interpret Inspect AI evaluation logs by extracting samples, messages, events, and performance metrics from .eval files, reducing manual log-parsing effort.
Install the Inspect AI package and load an evaluation file with the inspect_ai.log module, then iterate over logs and samples to compute key metrics.
High-intent search queries and answers about installing and using this skill.
To extract samples and metrics from Inspect AI evaluation logs, load .eval files using the inspect_ai.log module, iterate over samples and messages, and compute summary statistics or export results to dataframes.
Analyzing .eval files for per-sample model behavior requires iterating through evaluation runs, samples, and messages to inspect events and scores, which helps diagnose model responses and reduces manual log-parsing effort.
Yes, reading .eval files and generating dataframes requires the inspect-ai package and the inspect_ai.log analysis interfaces to load logs, iterate samples, and export structured results.
Yes, you can retrieve scores and task details from multiple evaluation runs for comparison by applying the analysis across tasks and models to enable per-sample inspection and summary statistics.
To convert Inspect AI evaluation data into a structured dataframe, load .eval files via the inspect_ai.log module, iterate over samples and events, and export the extracted metrics into a dataframe-ready format for analysis.
GroundMountCompanyAutomate AI image generation via Replicate API with model selection and iterative refinement.
AlterLab-IEUProcess out-of-core datasets with Vaex DataFrames for scalable analysis.
DouwMarxConstruct AI safety evaluation instruments using EquiStamp pipeline and Inspect framework.
kimon1230Manually compress Claude session progress entries with a custom Python compressor module.
