experimentation

To run reproducible LLM experiments, you provide a driving question with a measurable success criterion. The skill scaffolds a dated experiment folder containing raw outputs and a harness, ensuring your prompt and architecture testing is fully auditable.

The best way to document experiment evidence is by appending a new sprint record to your refactor documentation. This locks the validation results directly into your project timeline, tying token budget claims and architecture decisions to dated experiment outputs.

You test a new model integration by running a small, reproducible experiment that validates the specific NIM behavior. The process generates raw outputs and appends a sprint record, giving you decision-ready evidence before making a sprint commitment.

Yes, you can evaluate prompt scoring heuristics and latency claims by running targeted experiment runs. Each run generates a dated folder with raw outputs, turning ambiguous performance claims into reproducible evidence for your refactoring decisions.

Yes, the experimentation process improves future prompt evaluation quality through a self-improvement loop. It automatically appends concise lessons learned from each run back to the skill, enhancing subsequent architecture testing and experiment runs.