cost-aware-llm-pipeline

Implements model routing, budget tracking, retry logic, and prompt caching for LLM API cost control.

1|Updated Oct 11, 2025
One-click install
npx skills add https://github.com/ibytechaos/claude --skill cost-aware-llm-pipeline-ibytechaos
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: cost-aware-llm-pipeline
Source: https://github.com/ibytechaos/claude/tree/main/plugins/everything-claude-code/skills/cost-aware-llm-pipeline
Command: npx skills add https://github.com/ibytechaos/claude --skill cost-aware-llm-pipeline-ibytechaos

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic.

What problem does it solve? LLM API costs grow quickly when every request uses the most expensive model, retries fire on permanent errors, and long system prompts are resent on every call. This Skill provides composable patterns to control spend while preserving output quality on complex tasks. ## Core Features & Use Cases - Model Routing by Complexity: Automatically select cheaper models (e.g., Haiku) for simple tasks and reserve expensive models (e.g., Sonnet) for large inputs or high item counts. - Immutable Budget Tracking: Track cumulative spend with frozen dataclasses and fail early when a budget limit is exceeded. - Narrow Retry Logic: Retry only transient errors (rate limits, connection failures, server errors) with exponential backoff, failing fast on authentication or bad request errors. - Prompt Caching: Cache long system prompts with ephemeral cache control to cut both cost and latency. - Use Case: When batch-processing hundreds of documents through the Claude API, route each item to the right model, enforce a dollar budget, and cache the shared system prompt to keep total spend predictable. ## Quick Start Ask the AI to build a cost-aware LLM pipeline that routes requests between Haiku and Sonnet based on input size, tracks spending against a one-dollar budget, retries only transient errors, and caches the system prompt.

Frequently Asked Questions about cost-aware-llm-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM API costs in Python?

Route simple tasks to cheaper models like Haiku and reserve expensive models like Sonnet for complex inputs exceeding size or item-count thresholds. Combine this with prompt caching for long system prompts and strict budget limits to control total spend.

How to choose between Claude Haiku and Sonnet for a task?

Select the model based on measurable complexity signals such as text length and item count. For example, route inputs over 10,000 characters or 30 items to Sonnet, and everything else to Haiku, which costs roughly four times less.

Which Anthropic API errors should be retried?

Retry only transient errors: APIConnectionError, RateLimitError, and InternalServerError, using exponential backoff. Authentication and bad request errors are permanent and should fail immediately to avoid wasting budget.

Does prompt caching reduce Claude API costs?

Yes. Marking long system prompts with ephemeral cache_control avoids resending them on every request, reducing both token cost and latency. It is most effective for system prompts over 1024 tokens reused across many calls.

Why track LLM costs with immutable data structures?

Immutable trackers using frozen dataclasses return a new object on each recorded call instead of mutating state. This makes spend auditing and debugging straightforward and prevents accidental state corruption in concurrent pipelines.