model-verify-flagos

Debug multi-chip inference failures by running a base mode with the stack disabled and a full multi-chip mode, then diffing the outputs to pinpoint the exact failing layer. This isolates whether the issue stems from the model itself or the multi-chip integration.

Multi-chip model verification validates an AI serving stack by comparing a standard single-device run against a full multi-chip deployment. It identifies deployment regressions and locates the specific layer causing failures during heterogeneous hardware inference.

Locate the failing layer by executing a dual-run verification workflow that diffs the outputs of a base run and a full multi-chip run. This diff-based error localization highlights the precise integration point where the multi-chip stack diverges.

You need a container pre-configured with the AI serving stack to verify multi-chip inference. Users must also supply a model path inside the container and can optionally specify a tensor-parallel size to execute the verification workflows.

A model working in base mode but failing in multi-chip mode indicates an integration layer issue. The verification tool diagnoses this by diffing the outputs of both runs to isolate the specific layer where the multi-chip stack execution breaks down.

You can supply an optional tensor-parallel size when running the verification tool against your model path. This configures the full multi-chip run to match your target deployment topology during the diff-analysis.