mhc

Constrain Transformer residual updates with doubly stochastic matrices via Sinkhorn normalization.

1|Updated Jan 16, 2026

Search Tags:#transformers #mhc #manifold-constrained #doubly-stochastic #sinkhorn-knopp #residual-connections

Ready to run

npx skills add https://github.com/yonesuke/skills --skill mhc

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: mhc
Source: https://github.com/yonesuke/skills/tree/main/mhc
Command: npx skills add https://github.com/yonesuke/skills --skill mhc

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires jax, jaxlib, and includes scripts (resource) components.

What problem does it solve?

Deep Transformer residual connections can become unstable in very deep architectures; mHC uses manifold-constrained doubling to preserve signal flow and improve training stability.

Core Features & Use Cases

Applies a learned pre- and post-mapping to mix streams while preserving stable residuals.
Uses Sinkhorn normalization to constrain the residual mapping, enabling scalable deep models.
Use cases include training very deep Transformer models where gradient vanishing and representation collapse are concerns.

Quick Start

Apply mHC to stabilize very deep transformer networks by projecting the residual mapping onto the doubly stochastic manifold and using pre/post streams for controlled mixing.

Frequently Asked Questions about mhc

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I stabilize deep Transformer residual connections during training?▼

To stabilize deep Transformer residual connections, apply manifold-constrained doubling via Sinkhorn normalization to project residual mappings onto a doubly stochastic matrix, preserving signal flow and preventing representation collapse in very deep architectures.

Why does my deep Transformer model suffer from gradient vanishing and representation collapse?▼

Gradient vanishing and representation collapse in deep Transformers occur when residual connections become unstable. Constraining the residual update with doubly stochastic matrices through Sinkhorn normalization enforces stable signal mixing across very deep layers.

How do I apply doubly stochastic matrices to constrain Transformer residual updates?▼

To constrain Transformer residual updates with doubly stochastic matrices, apply a learned pre- and post-mapping to mix streams, then use Sinkhorn normalization to project the residual mapping onto the doubly stochastic manifold for controlled mixing.

Do I need a JAX environment to use Sinkhorn normalization for Transformer residual connections?▼

Yes, a JAX environment is required to implement Sinkhorn normalization for Transformer residual connections. The mHC Skill depends on both the jax and jaxlib libraries to execute the manifold-constrained operations for stabilizing deep architectures.

What is the best way to scale up very deep Transformer architectures without training instability?▼

The best way to scale very deep Transformer architectures without instability is using manifold-constrained doubling. This approach applies learned coefficients for pre/post mapping and Sinkhorn normalization to maintain stable residual signal flow.

When should I not use doubly stochastic matrix constraints for Transformer models?▼

Doubly stochastic matrix constraints for Transformer models are designed for very deep architectures. If your model is shallow or does not suffer from gradient vanishing or representation collapse, applying Sinkhorn normalization may introduce unnecessary computational overhead.