fiftyone-find-duplicates

Near-duplicate detection works by computing visual embeddings for your images and comparing them using a similarity threshold. Samples scoring above the threshold are grouped, allowing you to isolate visually similar images for review and removal.

You can deduplicate an image collection by running a brain-based similarity search to group visually similar samples. After reviewing the grouped duplicates in the FiftyOne App, you delete redundant samples to ensure a clean dataset for model training.

Yes, detected near-duplicates are isolated into saved views for guided review in the FiftyOne App. This allows you to visually inspect grouped duplicates and selectively delete redundant samples while keeping representative images.

Yes, computing embeddings is a required step. The Skill uses these visual embeddings to calculate brain-based similarity scores between images, which identifies near-duplicates when evaluated against a configured threshold.

The similarity threshold determines the strictness of near-duplicate grouping. A higher threshold requires images to be nearly identical, while a lower threshold groups more broadly similar samples, directly controlling the scope of the deduplication review.