cost-diff

Compares two cost-summary JSON snapshots to detect PR-level spend regressions.

70.1k|8.4k|Updated Jun 2, 2025
One-click install
npx skills add https://github.com/ruvnet/claude-flow --skill cost-diff
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: cost-diff
Source: https://github.com/ruvnet/claude-flow/tree/main/plugins/ruflo-cost-tracker/skills/cost-diff
Command: npx skills add https://github.com/ruvnet/claude-flow --skill cost-diff

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Teams running LLM workloads need to know whether a specific pull request increased API spend, but raw cost summaries alone cannot answer "what changed between these two snapshots?". This Skill computes the delta between two known-good cost-summary JSON outputs and fails CI when configurable thresholds are breached.

Core Features & Use Cases

  • Snapshot Delta Analysis: Compares baseline and current cost-summary JSON files per tier (haiku/sonnet/opus) and per model, tagging entries as added, removed, or changed, sorted by absolute delta.
  • PR-Gate Alerting: Exits non-zero when total spend grows beyond --alert-on-pct or --alert-on-usd thresholds, or when a token class (input, output, cache_write, cache_read) grows beyond --alert-on-class-pct limits.
  • Use Case: In CI, capture a cost summary on main as baseline.json, capture one on the PR branch as current.json, then run the diff with percent, dollar, and cache_write thresholds so any one signal failing blocks the merge.

Quick Start

Compare baseline.json and current.json cost snapshots and fail the PR if total spend grew more than 10 percent or 5 dollars.

Frequently Asked Questions about cost-diff

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect LLM cost regressions in a pull request?

Capture a cost summary JSON snapshot on the base branch and another on the PR branch, then run cost diff with --baseline and --current. Add --alert-on-pct or --alert-on-usd thresholds so the command exits 1 and fails the PR when spend grows too much.

How to compare two cost summary JSON snapshots?

Pass both files to cost diff via --baseline and --current. It validates the cost-summary shape, computes per-tier and per-model deltas tagged added, removed, or changed, and prints a table sorted by absolute delta so the biggest movers appear first.

What is the difference between cost-diff, cost-burn, and cost-counterfactual?

cost-diff compares two specific known-good snapshots, such as main versus a PR branch. cost-burn compares the latest bucket to the prior mean for trend detection, while cost-counterfactual compares against hypothetical always-haiku, always-sonnet, or always-opus baselines.

Can cost diff alert on specific token types like cache_write?

Yes, use --alert-on-class-pct with comma-separated class thresholds, for example cache_write:50,output:25. Valid classes are input, output, cache_write, and cache_read, and the first class to breach its threshold triggers exit code 1.

Why does cost diff exit with code 2?

Exit code 2 indicates a configuration error: missing snapshot files, invalid JSON, or snapshots lacking the required total_cost_usd and sessionCount fields. Exit 0 means no threshold fired and exit 1 means an alert threshold was exceeded.