skills.rest
shreyashankarshreyashankarCommunity·1 Agent Skills Included

error-discovery-skill

Error analysis, judge calibration, and eval pipeline audits

Guides teams through building product-specific evaluations for LLM pipelines, from trace review to judge calibration. Builds custom annotation interfaces, clusters traces for diverse sampling, and organizes reviewer notes into concrete failure modes. Audits existing eval setups, writes binary LLM-as-Judge prompts, and validates judges against human labels with TPR/TNR and bias correction.
211|19
↗
npx skills add shreyashankar/error-discovery-skill --all -g -y

All Skills in This Repository (1)

Pure Emerald Level Indicators
📦 In Repo
shreyashankarshreyashankar

error-discovery

Automate error analysis and data review on structured datasets.

Community
Intermediate

Frequently Asked Questions

FAQPage Schema
How to install error-discovery-skill?▼

Run `npx skills add shreyashankar/error-discovery-skill --all -g -y` in your terminal to install all skills in this suite globally.

How do I find failure modes in my LLM traces?▼

Point the error-discovery skill at a JSONL, CSV, or JSON file of traces. It builds a local review app, selects diverse samples, and organizes your notes into failure modes.

How do I know if my LLM judge is trustworthy?▼

Use the validate-evaluator skill to measure TPR and TNR against human labels on held-out data, then apply bias correction to production scores.

Can I audit an existing eval pipeline?▼

Yes. The eval-audit skill inspects your traces, judge prompts, and labeled data, then reports prioritized problems with concrete fixes.

Does this work with Claude Code and other coding agents?▼

Yes. All skills follow the standard SKILL.md format and run in Claude Code, Cursor, Codex, and other compatible agents.

Related Repositories in Data & Analytics

View All in Data & Analytics→
PaddlePaddlePaddlePaddle
87.3k

PaddleOCR

Extract text, tables, and formulas from PDFs and images

Official📦 2 Skills
D4VinciD4Vinci
73.3k

Scrapling

Scrape any website and bypass anti-bot protection with AI

Community📦 1 Skills
mvanhornmvanhorn
60.8k

last30days-skill

Research any topic across Reddit, X, YouTube, and the web

Community📦 2 Skills