cost-aware-llm-pipeline

Implements model routing, budget tracking, retry logic, and prompt caching for LLM API calls.

Updated Mar 18, 2026
One-click install
npx skills add https://github.com/freedom909/real-estate-saas --skill cost-aware-llm-pipeline-freedom909
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cost-aware-llm-pipeline
Source: https://github.com/freedom909/real-estate-saas/tree/main/.trae/skills/cost-aware-llm-pipeline
Command: npx skills add https://github.com/freedom909/real-estate-saas --skill cost-aware-llm-pipeline-freedom909

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic.

What problem does it solve? LLM API costs grow quickly when every request uses the most expensive model, retries fire on permanent errors, and long system prompts are resent on each call. This Skill provides composable patterns to control spend while preserving output quality on complex tasks. ## Core Features & Use Cases - Model Routing by Complexity: Automatically select cheaper models (e.g., Haiku) for simple tasks and reserve expensive models (e.g., Sonnet) for large inputs or high item counts. - Immutable Budget Tracking: Track cumulative spend with frozen dataclasses and fail early when a budget limit is exceeded. - Narrow Retry Logic: Retry only transient errors (connection, rate limit, server errors) with exponential backoff, failing fast on authentication or bad request errors. - Prompt Caching: Cache long system prompts with ephemeral cache control to cut both cost and latency. - Use Case: When batch-processing hundreds of product descriptions through Claude, route short items to Haiku, enforce a $1.00 budget cap, and cache the shared system prompt to reduce total spend. ## Quick Start Ask the AI to build a cost-aware LLM pipeline that routes requests between Haiku and Sonnet based on input size, tracks spend against a budget, and retries only transient API errors.

Frequently Asked Questions about cost-aware-llm-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM API costs in Python applications?▼

Reduce LLM API costs by routing simple tasks to cheaper models like Haiku, caching long system prompts with ephemeral cache control, and setting explicit budget limits before batch processing. Reserve expensive models only for inputs exceeding complexity thresholds.

How to route requests between Claude Haiku and Sonnet models?▼

Route requests by measuring input text length and item count against defined thresholds, such as 10,000 characters or 30 items. Inputs below the thresholds go to Haiku, which costs roughly 4x less, while larger inputs go to Sonnet.

Which API errors should be retried when calling Claude?▼

Retry only transient errors: APIConnectionError, RateLimitError, and InternalServerError, using exponential backoff. Authentication and bad request errors are permanent failures and should raise immediately to avoid wasting budget.

Does prompt caching work with the Anthropic API?▼

Yes, the Anthropic API supports prompt caching via cache_control with type ephemeral on message content blocks. Caching system prompts over 1024 tokens saves both cost and latency on repeated requests.

Why use immutable dataclasses for cost tracking?▼

Immutable frozen dataclasses ensure each API call returns a new tracker rather than mutating shared state. This makes debugging and auditing spend straightforward and prevents accidental state corruption in concurrent pipelines.

When should I not use model routing for LLM calls?▼

Avoid model routing when all tasks have uniform high complexity or when quality consistency matters more than cost. Routing adds threshold-tuning overhead, so log selection decisions and adjust thresholds based on real usage data.