segment-anything-model

Load a SAM checkpoint with sam_model_registry, create a SamPredictor, call set_image on your RGB image, then predict with point coordinates or a bounding box. The predictor returns masks with quality scores; select the highest-scoring mask.

ViT-B is the smallest (375MB) and fastest with good accuracy, ViT-L is medium (1.2GB), and ViT-H is the largest (2.4GB) and slowest but most accurate. Use ViT-B when GPU memory or speed is limited.

SAM alone only accepts geometric prompts like points and boxes, not text. Combine GroundingDINO for text-to-box detection with SAM for box-to-mask segmentation to achieve text-prompted segmentation.

The original SAM processes single images only and has no tracking. For video segmentation, use SAM 2, which adds a streaming memory architecture to propagate masks across frames.

The ViT-H model requires significant GPU memory, especially with large images. Switch to the ViT-B checkpoint, resize images to a maximum dimension, call torch.cuda.empty_cache() between images, or use half precision.

The default grid of points may be too sparse for small objects. Increase points_per_side, enable crop_n_layers for multi-scale processing, and lower min_mask_region_area to retain small masks.