Run strict TypeScript evals for LLM/agent workflows via CLI.
npx skills add https://github.com/lucasols/agent-eval --skill agent-eval-lucasols
Please help me install this Agent Skill. Skill: agent-eval Source: https://github.com/lucasols/agent-eval/tree/main/packages/cli/skills/agent-eval Command: npx skills add https://github.com/lucasols/agent-eval --skill agent-eval-lucasols
Teams building LLM/agent workflows struggle to define, run, and maintain repeatable evaluation harnesses. agent-eval provides a self-contained pattern where TypeScript evals live as *.eval.ts modules, discovered via agent-evals.config.ts, run through the CLI or UI, and persist results and traces to a local workspace.
Create a simple eval under evals/, configure agent-evals.config.ts, and run agent-evals run to execute and inspect results.
High-intent search queries and answers about installing and using this skill.
To run TypeScript evals for LLM agent workflows, you author strict TypeScript modules named *.eval.ts, configure them in agent-evals.config.ts, and execute via the CLI or app. Runs persist with traces, costs, and artifacts to a local workspace.
Maintaining repeatable evaluation harnesses for LLM agents involves authoring strict TS eval modules that persist traces, costs, and artifacts to a local workspace. This self-contained pattern enables reproducible runs and derived metrics for agent decision loops.
Yes, you can trace and account for costs across LLM agent evaluation runs. The CLI persists execution artifacts, traces, and cost accounting data to the .agent-evals/ directory, enabling cross-process inspection of agent decision loops.
Yes, TypeScript is required to use agent-eval for evaluating agent workflows. The Skill targets teams building evals against production-like flows by authoring strict TypeScript modules named *.eval.ts discovered from a configuration file.
The CLI discovers LLM agent evals through the agent-evals.config.ts file, which points to strict TypeScript modules named *.eval.ts. Once discovered, the CLI or app executes these modules and persists run artifacts locally.
labs21-devScaffold reusable skill packages with SKILL.md and starter eval resources.
Nick777-PixelInspect agent execution snapshots and trace LLM calls via CLI.
swaylqGuide LLM agent infrastructure decisions across frameworks, evals, observability, and RAG retrieval.
MesferAliEvaluate LLMs across benchmarks on Docker, Slurm, or cloud platforms.
