update-golden-values

To update golden values from a GitHub Actions workflow run, provide the workflow run ID and select a refresh scope. The skill downloads the CI artifacts and overwrites local golden-value JSON files with the new test outputs.

Yes, you can refresh golden values for failing tests by using the only-failing mode. This pulls artifacts exclusively from failing or cancelled jobs, rather than pulling artifacts from every job that produced goldens.

You can generate a PR summary for golden value changes by computing per-metric relative differences between old and new files. The skill aggregates these signed differences into headline statistics and a distribution table suitable for a PR description.

Relative-diff scoring for golden value artifacts measures the signed average relative difference between old and new values over shared steps. It calculates mean((old − new) / old) to quantify metric changes and bucket them by magnitude.

Yes, you need specific scripts to download and compare golden values. The process requires download_golden_values.py to fetch and overwrite local goldens, and compare_golden_values_kl.py to compute the signed avg_rel_diff and generate a CSV summary.

A limitation of refreshing Megatron-Core golden values is that the process specifically targets golden_values_*.json files within tests/functional_tests/test_cases. It requires a valid GitHub Actions workflow run ID to fetch the necessary CI artifacts.