baseline

Capture reproducible AI performance baselines for components before changes.

1|Updated Mar 26, 2026

Search Tags:#latency #benchmark #accuracy #baseline #cost #ai-evaluation

Ready to run

npx skills add https://github.com/selcukyucel/north-starr-genai --skill baseline-selcukyucel

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: baseline
Source: https://github.com/selcukyucel/north-starr-genai/tree/main/skills/baseline
Command: npx skills add https://github.com/selcukyucel/north-starr-genai --skill baseline-selcukyucel

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Capture a reproducible AI performance baseline for a given component before changes, ensuring stakeholders can compare post-change results against a known reference.

Core Features & Use Cases

Dispatches the baseline-capturer agent on a separate thread to avoid blocking workflows.
Measures accuracy, latency, cost, and error rate by analyzing the codebase and any existing eval suites or test fixtures.
Writes reproduction steps to .plans/BASELINE-<name>.md and coordinates with related agents (eval-designer, cost-estimator, ai-ops) as needed.
Use Case: Before releasing changes that affect client-facing AI outputs, run this baseline to establish a reference point.

Quick Start

Provide the component or pipeline name to baseline, and let the baseline-capturer run on a separate thread.

Frequently Asked Questions about baseline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I capture an AI performance baseline before deploying code changes?▼

To capture an AI performance baseline, this Skill automates measuring accuracy, latency, cost, and error rate for a component. It dispatches a background agent to analyze the codebase and existing eval suites, then writes reproduction steps to a .plans/BASELINE-<name>.md file.

What metrics are included when benchmarking AI model components?▼

Benchmarking AI model components measures accuracy, latency, cost, and error rate. The Skill evaluates these dimensions by analyzing your codebase alongside any existing test fixtures or eval suites to establish a reliable reference point.

Can I establish an AI evaluation baseline without blocking my active workflows?▼

Yes, you can establish an AI evaluation baseline without blocking workflows. The Skill spawns a baseline-capturer agent on a separate background thread, allowing your development and deployment pipelines to continue running uninterrupted.

How do I reproduce AI evaluation results after modifying a client-facing pipeline?▼

To reproduce AI evaluation results after modifications, the Skill generates a .plans/BASELINE-<name>.md file containing detailed reproduction steps. This ensures stakeholders can compare post-change outputs against the known reference.

Does capturing an AI latency baseline require existing eval suites?▼

Capturing an AI latency baseline does not strictly require existing eval suites. The baseline-capturer analyzes the codebase and any available test fixtures, coordinating with related agents like eval-designer and cost-estimator to measure performance comprehensively.

What is the best way to evaluate AI accuracy and cost before a release?▼

The best way to evaluate AI accuracy and cost before a release is running an automated baseline capture. It measures performance metrics against current code and eval suites, creating a reproducible reference document for post-change comparison.