segment-anything-model

Load a checkpoint with sam_model_registry, create a SamPredictor, call set_image once, then predict with point or box prompts. The predict method returns masks, scores, and logits; select the mask with the highest score.

ViT-B is the smallest at 375MB and fastest, ViT-L is medium at 1.2GB, and ViT-H is the largest at 2.4GB with the best accuracy. Choose based on your GPU memory and speed requirements.

SAM alone does not accept text prompts; it requires points, boxes, or masks. Combine GroundingDINO with SAM to convert text descriptions into bounding boxes, then feed those boxes to SAM for mask generation.

SAM can run on CPU by not calling .to("cuda"), but inference is significantly slower, especially for the image encoder. For limited GPU memory, use the ViT-B model or export to ONNX for optimized deployment.

CUDA out of memory occurs with large models like ViT-H or large images. Switch to ViT-B, reduce image resolution, call torch.cuda.empty_cache() between images, or use half precision with sam.half().

Use SAM 2 for video segmentation tasks, since it adds streaming memory and object tracking across frames. Standard SAM processes individual images only and has no temporal awareness.