skills.rest
Lingtai-AI
Lingtai-AIOFFICIAL
Explore all skills by this author →
GitHub

dev-guide-cache-hit-rate

Computes prompt-cache hit rates from LingTai token ledgers over rolling time windows.

692|60|Updated Mar 15, 2026
Search Tags:#JSONL#prompt caching#token ledger#python script#lingtai#LLM observability#cache hit rate
One-click install
npx skills add https://github.com/Lingtai-AI/lingtai --skill dev-guide-cache-hit-rate
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dev-guide-cache-hit-rate
Source: https://github.com/Lingtai-AI/lingtai/tree/main/tui/internal/preset/skills/lingtai-dev-guide/reference/cache-hit-rate
Command: npx skills add https://github.com/Lingtai-AI/lingtai --skill dev-guide-cache-hit-rate

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

When prompt caching is configured for LLM calls, it is hard to know whether it is actually working. This Skill measures the real cache hit rate from LingTai's append-only token ledgers, so you can verify cache effectiveness or diagnose a drop after a refresh, affinity, or cache-key change.

Core Features & Use Cases

  • Rolling-window metrics: Computes hit rate as sum(cached)/sum(input) over default windows of 1h, 5h, 1d, and 3d, with custom window support.
  • Double-count protection: Resolves agent workdirs, project roots, or single ledger files while avoiding double-counting daemon calls that appear in two ledgers.
  • Read-only stdlib script: Ships scripts/cache_hit_rate.py, a deterministic Python script using only the standard library, with JSON output, source filtering, and a pinnable clock for reproducible runs.
  • Use Case: After deploying a cache-affinity fix, run the script against .lingtai/codex to confirm the hit rate recovered, instead of guessing from provider dashboards.

Quick Start

Ask the agent to run the cache hit rate script against the current agent workdir or project .lingtai directory and report the 1h, 5h, 1d, and 3d hit rates.

Frequently Asked Questions about dev-guide-cache-hit-rate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure prompt cache hit rate for LLM API calls?▼

Run scripts/cache_hit_rate.py against an agent workdir, project root, or a single token_ledger.jsonl file. It computes sum(cached)/sum(input) over rolling windows (default 1h, 5h, 1d, 3d) and prints a table or JSON report.

How to check if prompt caching is working after a config change?▼

Run the script against the agent's .lingtai/<agent> directory and compare hit rates across windows before and after the change. The token ledger is the observable proof that a cache or affinity fix actually took effect.

Does the cache hit rate script work with different LLM providers?▼

Yes, the metric is provider-agnostic. The LingTai kernel normalizes every provider's usage into the same input and cached fields before writing the ledger, verified across OpenAI, DeepSeek, and Anthropic adapters.

Why does scanning a project directory double-count daemon token usage?▼

Daemon LLM calls are written to both the daemon's own ledger and the parent agent's ledger. The script avoids this by reading only each agent's top-level logs/token_ledger.jsonl and never recursing into daemons/ directories.

What Python dependencies does the cache hit rate script need?▼

None beyond the standard library. The script uses only argparse, json, datetime, and pathlib, and runs on any Python 3.11+ interpreter without installing packages.

Why is my cache hit rate showing n/a for all windows?▼

An n/a result means no input tokens fell within the windows, typically because the agent was idle or a pinned --now timestamp predates the activity. Widen the window, drop source filters, or check the ledger's timestamp range.

Related Skills

Explore More Software Engineering Skills→
sgl-projectsgl-project
33.0k

debug-cuda-crash

Diagnose CUDA crashes in SGLang using kernel API logging and crash-safe tensor dumps.

Official
Intermediate
NovaAI-innovationNovaAI-innovation

Optimizing Cache Performance

Analyze Redis cache configurations and recommend TTL, key, and invalidation optimizations.

Community
Intermediate
PostHogPostHog
713

investigating-metric-anomalies

Diagnoses server metric anomalies by correlating metrics, logs, and traces to root causes.

Official
Advanced
BedaBeda-GrowthBedaBeda-Growth

List Rules

Display Company Rule Ledger rules from a local cache with filtering.

Official
Intermediate