skills.rest
malinovskiy-makar
malinovskiy-makarCOMMUNITY
Explore all skills by this author →
GitHub

cost-aware-llm-pipeline

Implements model routing, budget tracking, retry logic, and prompt caching for LLM API cost control.

Updated Jun 13, 2026
Search Tags:#llm#retry logic#cost optimization#model routing#anthropic#budget tracking#prompt caching
One-click install
npx skills add https://github.com/malinovskiy-makar/qls --skill cost-aware-llm-pipeline-malinovskiy-makar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cost-aware-llm-pipeline
Source: https://github.com/malinovskiy-makar/qls/tree/main/.claude/skills/cost-aware-llm-pipeline
Command: npx skills add https://github.com/malinovskiy-makar/qls --skill cost-aware-llm-pipeline-malinovskiy-makar

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic.

What problem does it solve? LLM API spend grows quickly when every request uses the most expensive model, retries fire on permanent errors, and long system prompts are resent on every call. This Skill provides composable patterns to route tasks across model tiers, enforce budget limits, retry only transient failures, and cache prompts. ## Core Features & Use Cases - Model Routing by Complexity: Automatically select cheaper models (e.g., Haiku) for simple tasks and reserve expensive models (e.g., Sonnet) for complex ones based on text length and item count thresholds. - Immutable Budget Tracking: Track cumulative spend with frozen dataclasses and fail early when a budget limit is exceeded. - Narrow Retry Logic: Retry only transient errors (rate limits, connection failures, server errors) with exponential backoff while failing fast on authentication or bad request errors. - Prompt Caching: Cache long system prompts with ephemeral cache control to reduce token costs and latency. - Use Case: A batch pipeline processing thousands of documents routes short items to Haiku, long ones to Sonnet, checks a $1.00 budget before each call, and logs every model selection decision for threshold tuning. ## Quick Start Ask the AI to build a cost-aware LLM pipeline that routes requests between Haiku and Sonnet based on task complexity, tracks spend against a budget, retries only transient errors, and caches the system prompt.

Frequently Asked Questions about cost-aware-llm-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM API costs in a Python application?▼

Route simple tasks to cheaper models like Haiku and reserve expensive models like Sonnet for complex tasks based on text length or item count thresholds. Combine this with prompt caching for long system prompts and budget checks before each API call.

How to route requests between Claude Haiku and Sonnet models?▼

Define thresholds such as 10,000 characters of text or 30 items per batch, then select Sonnet when either threshold is exceeded and Haiku otherwise. Allow a force_model parameter to override the automatic selection when needed.

Which API errors should be retried for LLM calls?▼

Retry only transient errors: APIConnectionError, RateLimitError, and InternalServerError, using exponential backoff with a maximum of three attempts. Authentication and bad request errors should raise immediately since retrying them wastes budget.

Does prompt caching work with the Anthropic API?▼

Yes, add cache_control with type ephemeral to the system prompt content block so it is cached across requests. This saves both cost and latency for system prompts over 1024 tokens that are resent on every call.

Why use immutable dataclasses for cost tracking?▼

Frozen dataclasses ensure each API call returns a new tracker instead of mutating shared state, which makes debugging and auditing spend straightforward. The tracker exposes total_cost and over_budget properties computed from its record tuple.

When should I not use model routing for LLM costs?▼

Avoid routing when all tasks have uniform complexity, since threshold logic adds overhead without savings. Also avoid hardcoding model names throughout the codebase; use constants or configuration so pricing and model updates stay maintainable.

Related Skills

Explore More Software Engineering Skills→
Fabio29TFabio29T

cost-aware-llm-pipeline

Route LLM API calls by cost with budget tracking, retries, and prompt caching.

Community
Advanced
Habitat-ThinkingHabitat-Thinking
44

cost-tracking

Capture AI provider spend and token usage into structured Markdown cost snapshots.

Official
Intermediate
crissdiamondcrissdiamond

crisai-eval-baseline-campaign

Evaluate LLM product quality across routing, grounding, fidelity, and cost metrics.

Community
Advanced
hbvg234hbvg234
1

llm-cost-optimizer

Reduce LLM API spend with model routing, caching, and prompt compression.

Community
Advanced