skills.rest
ruvnet
ruvnetCOMMUNITY
Explore all skills by this author →
Website𝕏@ruvGitHub

cost-trend

Analyzes persisted benchmark run JSON files to detect drift in win rate, latency, and cost.

70.1k|8.4k|Updated Jun 2, 2025
Search Tags:#latency#benchmark#cost tracking#trend analysis#regression detection#Nodejs#win rate
One-click install
npx skills add https://github.com/ruvnet/claude-flow --skill cost-trend
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cost-trend
Source: https://github.com/ruvnet/claude-flow/tree/main/plugins/ruflo-cost-tracker/skills/cost-trend
Command: npx skills add https://github.com/ruvnet/claude-flow --skill cost-trend

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Binary pass/fail smoke gates miss slow regressions, such as win rate gradually declining from 100% to 85% while still passing. This Skill reads every persisted benchmark run and surfaces drift in win rate, latency, escalation rate, and LLM-baseline cost over time.

Core Features & Use Cases

  • Trend Analysis: Reads all runs in docs/benchmarks/runs/*.json and reports first-to-last deltas plus a per-run series.
  • Regression Flagging: Emits warnings when win rate drops between first and last run or average latency rises 1.5x or more.
  • Use Case: Before a release, run the trend script to confirm the speedup versus the Gemini baseline has not drifted, or after upgrading agent-booster to surface latency and strategy changes.

Quick Start

Run the cost-trend skill to analyze all benchmark runs in docs/benchmarks/runs and report any regressions in win rate or latency.

Frequently Asked Questions about cost-trend

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I track benchmark performance trends over time?▼

Run node plugins/ruflo-cost-tracker/scripts/trend.mjs from the project root. It reads every run in docs/benchmarks/runs/*.json and reports first-to-last deltas plus a per-run series for win rate, latency, and escalation rate.

How to detect regressions that pass/fail smoke tests miss?▼

Use trend analysis across persisted runs instead of a binary gate. The script flags regressions when win rate drops between first and last run or average latency rises 1.5x or more from the first run.

Can I output benchmark trend results as JSON?▼

Yes, set TREND_FORMAT=json as an environment variable to emit JSON instead of markdown. You can also set TREND_LIMIT=10 to consider only the most recent N runs.

What benchmark data does the trend analysis require?▼

It requires persisted run JSON files in docs/benchmarks/runs/, produced by the cost-benchmark skill. Each run records the corpus version, so trends across corpus versions remain interpretable.

When should I check benchmark trends?▼

Check trends before a release to confirm speedup has not drifted, after expanding the benchmark corpus to verify older runs still hold, and after upgrading agent-booster to surface latency or strategy changes.

Related Skills

Explore More Software Engineering Skills→
faroukBakarifaroukBakari
5

model-selection

Compare Opus, Sonnet, Haiku, Claude, GPT, and Gemini models by benchmark data and cost multipliers.

Community
Intermediate
ryan-nguyen-01ryan-nguyen-01
4

agent-qa

Coordinate QA strategy and release sign-off for pre-release activities.

Community
Advanced
X-McKayX-McKay
3

detect-metric-anomaly

Detect statistical anomalies in time-series metrics using z-score calculations and thresholds.

Community
Intermediate
BearflowerBearflower

timescaledb

Extend PostgreSQL with TimescaleDB hypertables and continuous aggregates for time-series analytics.

Community
Advanced