eval-collector

To generate eval JSON from git diffs after an SDD workflow, you execute the collect.py script. This script analyzes directory changes, extracts feature_infos from tasks.md, and builds code_blocks from git diffs to produce an evaluation-ready JSON artifact.

The evaluation-ready JSON payload follows an api and input structure suitable for downstream automated evaluation. It is saved as config.result.json under the changes/<branch>/evalset directory, containing extracted feature_infos and code_blocks.

The tool extracts feature_infos from tasks.md by locating and parsing lines labeled with the purpose description. These extracted lines are then included in the generated JSON payload to provide context for the code changes.

You do not need specific external dependencies to collect code change information for eval JSON. The skill operates using internal scripts and relies on standard git diff outputs and the presence of a tasks.md file in the target directory.

The generated evaluation JSON is saved directly to the changes/<branch>/evalset/config.result.json path. This ensures the evaluation payload is organized by branch and ready for automated evaluation systems to access.