project-development

LLM-suited tasks involve synthesis across sources, subjective judgment with rubrics, natural language output, and tolerance for occasional errors. Tasks requiring precise computation, real-time responses, or deterministic output are poor fits. Validate by manually testing one representative example with the target model before building automation.

Use five discrete stages: acquire, prepare, process, parse, render. Store intermediate outputs as files per item (raw.json, prompt.md, response.md, parsed.json) so file existence gates execution. This gives idempotency, easy debugging, and cheap re-runs of only the expensive LLM stage.

Use multi-agent architectures when tasks exceed a single context window or require parallel exploration of independent subtasks. The main benefit is context isolation, not role specialization. Single pipelines are simpler and cheaper for batch processing of independent items.

Multiply item count by tokens per item by price per token, then add 20-30% buffer for retries and failures. Estimate input tokens (prompt plus context) and output tokens separately, and track actual costs during development to catch overruns early.

LLMs do not follow format instructions perfectly, so rigid parsers break on minor variations. Specify exact formats with examples in the prompt, state that output will be parsed programmatically, and build parsers with flexible regex, sensible defaults, and failure logging instead of crashes.

Reduction outperforms complexity when your data layer is well-documented and the model has strong reasoning ability. Vercel's d0 agent improved from 80% to 100% success by cutting 17 specialized tools down to bash and SQL. Keep specialized tools only for messy data or safety constraints.