Verified:GitHub

bump-base-image

Update GitHub and GitLab CI container pins for the Megatron-LM NGC PyTorch base image.

Updated May 23, 2026

Search Tags:#github actions #docker #ci #megatron-core #golden values #ngc pytorch #gitlab pipeline

One-click install

npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill bump-base-image

Or copy as Structured Prompt for Agent▼

Please help me install this Agent Skill.
Skill: bump-base-image
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/Megatron-Core/bump-base-image
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill bump-base-image

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Prevents CI breakage when upgrading Megatron-LM’s nvcr.io/nvidia/pytorch:<YY.MM>-py3 container by ensuring both GitHub and GitLab Docker pins are updated together and by guiding remediation steps after the bump.

Core Features & Use Cases

Dual pin bump (GitHub + GitLab): Updates docker/.ngc_version.dev for GitHub CI and the BASE_IMAGE matrix rows for GitLab CI so both pipelines use the same container tag.
Post-bump functional test handling: Recommends re-running functional tests, refreshing golden values when expected drift occurs, and isolating genuine regressions without blocking the bump.
Operational gotchas and guardrails: Addresses common failure modes like mismatched pins, hangs/timeouts/OOM after the bump, and per-commit /ok to test <sha> requirements for fork PRs.
Use case: You want to upgrade CI from 25.xx-py3 to 26.04-py3, but GitLab fails on main because the GitHub-only pin was updated first.

Quick Start

Tell the assistant the target NGC tag and whether you’re bumping only dev (e.g., “bump base image to 26.04-py3 for dev”), and follow it to update both CI pins and run the functional test workflow with golden-value refresh or broken-scope marking as needed.

Frequently Asked Questions about bump-base-image

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I safely bump the NGC PyTorch base image in Megatron-LM CI?▼

To safely bump the NGC PyTorch base image, update both the GitHub CI pin in `docker/.ngc_version.dev` and the GitLab CI `BASE_IMAGE` matrix rows in `.gitlab/stages/01.build.yml` so both pipelines use the same container tag.

Why does GitLab CI break when I update the NGC PyTorch image in Megatron-LM?▼

GitLab CI breaks when the NGC PyTorch image is updated because mismatched container pins occur if GitHub and GitLab CI are not updated together, causing pipeline failures on `main` from version drift.

What should I do if functional tests hang, time out, or OOM after a PyTorch container upgrade?▼

If functional tests hang, time out, or OOM after a PyTorch container upgrade, re-run the functional tests, refresh golden values for expected drift, and isolate genuine regressions using `mr-broken` scope changes to avoid blocking the bump.

Do I need to update both GitHub and GitLab CI to switch the Megatron-LM nvcr.io container?▼

Yes, switching the Megatron-LM nvcr.io container requires updating both GitHub and GitLab CI pins to prevent mismatched tags and ensure both pipelines execute against the same `nvcr.io/nvidia/pytorch:<YY.MM>-py3` image.

How do I handle golden value drift when upgrading the Megatron-LM CI container?▼

To handle golden value drift when upgrading the Megatron-LM CI container, refresh the golden values after re-running functional tests to account for expected changes from the new NGC PyTorch image.

What are the limitations of bumping the Megatron-LM base image for fork PRs?▼

A limitation of bumping the Megatron-LM base image for fork PRs is the per-commit `/ok to test <sha>` requirement, which must be addressed to trigger CI pipelines successfully on external contributions.