ai-engineering

Map every input that reaches the prompt and mark which ones third parties can write, then check each tool for reversibility and idempotency, verify turn cap, deadline, and spend cap exist, and confirm retries wrap only the model call. Each finding must name the observable failure it produces.

Separate the retrieval unit from the storage unit by chunking documents, set a relevance floor instead of top-k alone, assert index and query embeddings use the same model and dimension, and place chunks in a labelled data region with citation ids required in the answer.

An average-score gate cannot be moved by one catastrophic case among hundreds, so regressions pass. Use per-case pass/fail with a named must-never-fail set, assert code-checkable properties instead of asking a judge, and fail the case when the judge reply is unparseable.

No. Under strict schema mode every property becomes required and language-level defaults stop applying, so an Optional field is emitted as null on every call. Code testing key membership instead of the value will crash on NoneType.

Skip the loop when the steps are known in advance, one call with a schema would do, the task is classification, or determinism is required. A fixed sequence of calls is cheaper, faster, and removes the chance of the model picking the wrong tool.

The four-characters-per-token heuristic underestimates JSON records by about 41% and identifier-heavy payloads by about 67%, which is exactly the content tool results contain. Measure with the provider's tokenizer, count endpoint, or the usage numbers from the previous response.