evals-live-run

Run repeatable live-device skill evaluations by using an eval harness that accepts device serials and operator packages to orchestrate controlled debugging sessions. This process targets real devices to produce consistent artifacts and logs for analysis.

Retained-log debugging is a workflow that enables selective replay of the newest evaluation batches. It works by retaining logs produced during live-device runs, allowing operators to debug issues using the preserved artifacts after the evaluation completes.

Yes, you can target specific devices when running Solax cold-start proving evaluations. The evaluation harness accepts explicit device serials, ensuring the orchestrated cold-start runs execute against your intended real hardware.

Yes, the live-device evaluation harness supports Android version benchmarks for Samsung devices. It provides a wrapper to run Pack A Samsung benchmarks and other evaluation scenarios with explicit device targeting for accurate results.

Yes, you need specific operator packages to start a live-device debugging session. The evaluation harness requires these packages along with device serials to properly orchestrate repeatable skill proving runs across real devices.

A limitation when replaying retained logs is that selective replay is restricted to the newest evaluation batches. You cannot replay arbitrary historical logs; the system focuses on recent live-device debugging sessions for targeted analysis.