mhc-algorithm

What problem does it solve?

Stabilizes deep network training by constraining residual stream mixing with a doubly stochastic matrix and using Sinkhorn-Knopp projections for stable optimization.

Core Features & Use Cases

• HyperConnections module wraps attention or MLP sublayers to enable multi-stream processing
• Sinkhorn-Knopp projection enforces doubly stochastic H_res for gradient stability
• Works with deep Transformers and residual architectures to improve convergence in challenging training regimes

Quick Start

Initialize HyperConnections with a chosen number of residual streams and wrap the target sublayer (attention or MLP), then begin training to observe improved stability.