model-verify-flagos

What problem does it solve?

Verify the serving stack against a user-specified model by running two verification workflows: a base run with the multi-chip stack disabled to isolate model-specific errors, and a full run with the multi-chip stack enabled. The tool diffs the two runs to pinpoint which layer caused a failure.

Core Features & Use Cases

• Dual-run verification against a target model to validate base vs multi-chip deployments.
• Diff-based error localization to identify whether issues originate from the model, the multi-chip stack, or the integration layer.
• Use cases include validating inference serving on heterogeneous hardware, diagnosing deployment regressions, and ensuring reliable multi-GPU operation before production testing.

Quick Start

Provide a model path inside the container and run the verification tool to compare base and multi-chip runs.