kermt-add-cmim-pretrain

To upgrade a GROVER checkpoint to a hybrid cMIM model, convert the legacy grover_base checkpoint by adding a cMIM decoder and latent distribution. This enables continued pretraining on custom corpora using both vocabulary and contrastive objectives without retraining from scratch.

Adding a contrastive objective to a biomolecular language model allows you to integrate SMILES-reconstruction contrastive learning. This avoids the need for costly from-scratch pretraining while extending existing encoder-only checkpoints into hybrid architectures.

Yes, converting legacy grover_base checkpoints requires a CUDA-capable NVIDIA GPU environment. You must also have the nvidia-container-toolkit installed and pre-configured kermt repository dependencies to execute the conversion scripts.

Yes, you can use a custom corpus for hybrid pretraining after checkpoint conversion. The upgraded hybrid cMIM model supports continued pretraining on custom corpora using both vocabulary and contrastive objectives simultaneously.

The best way to avoid costly from-scratch pretraining is upgrading existing grover_base checkpoints to a hybrid architecture. This approach adds a cMIM decoder and latent distribution directly, enabling SMILES-reconstruction contrastive learning without days of retraining from random initialization.