What problem does it solve?
This Skill prevents CI breakage when upgrading Megatron-LM’s NVIDIA PyTorch container base image by ensuring both GitHub and GitLab CI pins are updated together and that downstream functional-test expectations stay consistent.
Core Features & Use Cases
- Dual-pin container bump: Updates the GitHub CI tag in
docker/.ngc_version.dev and the GitLab CI BASE_IMAGE tag in .gitlab/stages/01.build.yml for both amd64 and arm64, avoiding the common “one CI updated, the other still old” failure mode.
- CI validation loop guidance: Recommends re-running functional tests, handling golden-value drift, and responding appropriately to hangs/OOMs or true regressions instead of conflating them with the container change.
- Recipe scope handling: Provides the exact
mr-broken / mr-github-broken scope mapping to keep broken cases runnable without blocking PR merges.
Quick Start
Use the bump-base-image skill to upgrade the CI container tag (for example from the current nvcr.io/nvidia/pytorch:<YY.MM>-py3 to a new <YY.MM>-py3) and then refresh golden values if functional metrics drift.