skills.rest
Femad-6
Femad-6COMMUNITY
Explore all skills by this author →
WebsiteGitHub

cost-aware-llm-pipeline

Implements model routing, budget tracking, retry logic, and prompt caching for LLM API cost control.

Updated Mar 25, 2026
Search Tags:#llm#retry logic#cost optimization#model routing#anthropic#budget tracking#prompt caching
One-click install
npx skills add https://github.com/Femad-6/my-skills --skill cost-aware-llm-pipeline-femad-6
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cost-aware-llm-pipeline
Source: https://github.com/Femad-6/my-skills/tree/main/.github/skills/cost-aware-llm-pipeline
Command: npx skills add https://github.com/Femad-6/my-skills --skill cost-aware-llm-pipeline-femad-6

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic.

What problem does it solve? LLM API costs grow quickly when every request uses the most expensive model, retries fire on permanent errors, and long system prompts are resent on every call. This Skill provides composable patterns to control spend without sacrificing quality on complex tasks. ## Core Features & Use Cases - Model Routing by Complexity: Automatically select cheaper models (e.g., Haiku) for simple tasks and reserve expensive models (e.g., Sonnet) for large inputs or high item counts. - Immutable Budget Tracking: Track cumulative spend with frozen dataclasses and fail early when a budget limit is exceeded. - Narrow Retry Logic: Retry only transient errors (rate limits, connection failures, server errors) with exponential backoff, failing fast on authentication or bad request errors. - Prompt Caching: Cache long system prompts with ephemeral cache control to reduce token costs and latency. - Use Case: When processing a batch of 500 documents through the Claude API, route small documents to Haiku, enforce a $1.00 budget cap, and cache the shared system prompt to cut total spend by roughly 4x. ## Quick Start Ask the AI to build a cost-aware LLM pipeline that routes requests between Haiku and Sonnet based on input size, tracks spend against a budget, and caches the system prompt.

Frequently Asked Questions about cost-aware-llm-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM API costs in Python?▼

Reduce LLM API costs by routing simple tasks to cheaper models like Haiku, caching long system prompts with ephemeral cache control, and enforcing budget limits before processing batches. Combining these techniques can cut spend by 4x or more compared to using one expensive model.

How to route requests between Claude Haiku and Sonnet models?▼

Route requests by setting complexity thresholds on input size and item count. For example, send text under 10,000 characters and fewer than 30 items to Haiku, and larger workloads to Sonnet, using a select_model function with an optional force_model override.

Which Anthropic API errors should be retried?▼

Retry only transient errors: APIConnectionError, RateLimitError, and InternalServerError, using exponential backoff with a maximum of three attempts. AuthenticationError and BadRequestError indicate permanent failures and should raise immediately to avoid wasting budget.

Does prompt caching work with the Anthropic API?▼

Yes, the Anthropic API supports prompt caching by adding a cache_control field with type ephemeral to message content blocks. Caching system prompts over 1024 tokens reduces both cost and latency on repeated requests with the same prompt.

Why use immutable dataclasses for cost tracking?▼

Immutable frozen dataclasses ensure each API call returns a new tracker instead of mutating shared state, making spend auditing and debugging straightforward. The tracker exposes total_cost and over_budget properties so pipelines can fail early when limits are exceeded.

Related Skills

Explore More Software Engineering Skills→
Syedyasir001Syedyasir001

project-development

Design and validate LLM batch processing pipelines with staged architecture and cost estimation.

Community
Advanced
Habitat-ThinkingHabitat-Thinking
44

cost-tracking

Capture AI provider spend and token usage into structured Markdown cost snapshots.

Official
Intermediate
BagelHoleBagelHole
46

rag-observability-evals

Monitor RAG systems with retrieval and generation quality metrics.

Community
Advanced
SeanChenRSeanChenR

cost-aware-llm-pipeline

Route LLM API tasks to cost-appropriate models and track spend.

Community
Advanced