Convert raw agent transcripts into structured JudgeInput and ai_annotation data.
npx skills add https://github.com/haizelabs/annotate --skill annotate
Please help me install this Agent Skill. Skill: annotate Source: https://github.com/haizelabs/annotate/tree/main/annotate_skill Command: npx skills add https://github.com/haizelabs/annotate --skill annotate
Tired of manual AI log analysis and complex evaluation setups? This Skill automates the intricate process of evaluating AI agent performance, extracting key data, and setting up LLM-as-a-judge systems. Save significant time and reduce complexity in your AI development lifecycle, allowing you to focus on improving your models.
Generate the boilerplate ingestion script to prepare your raw AI agent logs for annotation and evaluation.
High-intent search queries and answers about installing and using this skill.
Convert agent transcripts into structured judge inputs by loading raw data, applying summarization with optional context and attribute matchers, and producing normalized JudgeInput objects ready for AI evaluation and annotation workflows.
Define precise evaluation criteria and feedback configurations via dedicated APIs, then use the React frontend and FastAPI backend to annotate transcripts at step, interaction, or group granularities—supporting both pointwise and ranking test cases.
Yes. Ingest agent conversation logs, configure rubrics for specific evaluation dimensions like helpfulness, then efficiently collect human and AI annotations across multiple interaction types and granularity levels in a single workflow.
Transform raw conversation logs into annotated training data by extracting interactions, defining feedback criteria through the API, and generating structured annotations that capture performance signals for downstream model improvement.
Supports ingestion of raw agent transcripts, extraction and validation at multiple granularities—steps, interactions, and groups—with flexible feedback configurations and output as structured JudgeInput and ai_annotation objects for evaluation pipelines.
No. The skill automates the setup complexity; you define evaluation criteria through the API and use the interactive React interface to annotate transcripts, letting the system handle the LLM-as-a-judge logic and data normalization.
jesseotremblayTransform, clean, and extract structured information from CSV and JSON data.
0xMerl99Guide fine-tuning of large language models with LoRA and QLoRA.
Zack2626-okBuild full-stack Databricks Apps with React and FastAPI.
himanshu231204Benchmark coding agents with reproducible metrics and git worktree isolation.
