skills.rest
MLT-OSS
MLT-OSSOFFICIAL
Explore all skills by this author →
WebsiteGitHub

clip

Performs zero-shot image classification and image-text similarity using OpenAI's CLIP model.

14|5|Updated Apr 9, 2026
Search Tags:#embeddings#image search#pytorch#content moderation#clip#vision-language#zero-shot classification
One-click install
npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill clip-mlt-oss
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/MLT-OSS/hermes-agent-go/tree/main/skills/mlops/models/clip
Command: npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill clip-mlt-oss

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? Classifying images or searching image collections normally requires labeled training data and custom model training. CLIP removes that requirement by matching images to natural-language descriptions, enabling zero-shot classification, semantic image search, and content moderation without fine-tuning. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text-defined categories with no training data, using models from RN50 up to ViT-L/14. - Semantic Image Search & Retrieval: Encode images and text into a shared embedding space for cross-modal search, and integrate embeddings with vector databases like Chroma or FAISS. - Content Moderation: Score images against safety categories such as NSFW or violent content using softmax probabilities. - Use Case: Index a folder of product photos with CLIP embeddings, then let users search the catalog with queries like "a red sneaker on white background" and retrieve the top matches by cosine similarity. ## Quick Start Use the clip skill to classify the attached photo against the labels dog, cat, bird, and car and show the probability for each.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then pass both through the model and apply softmax to the logits. Descriptive labels like "a photo of a dog" perform better than single words.

How to build semantic image search with CLIP embeddings?▼

Encode all images with model.encode_image, normalize the embeddings, then encode a text query and compute cosine similarity against the image embeddings. Store embeddings in Chroma or FAISS to query large collections efficiently.

Which CLIP model should I use: RN50, ViT-B/32, or ViT-L/14?▼

ViT-B/32 is the recommended default, balancing speed and quality at 151M parameters. RN50 is fastest, while ViT-L/14 gives the best quality but is the slowest at 428M parameters.

Does CLIP work on CPU or does it require a GPU?▼

CLIP runs on CPU, with image encoding taking roughly 200ms per image. A GPU is recommended since it speeds encoding by 10-50x, bringing image encoding down to about 20ms on a V100.

What are the limitations of CLIP for image understanding?▼

CLIP works best for broad categories, not fine-grained tasks. It produces no bounding boxes, has weak spatial and counting understanding, can reflect biases from web training data, and needs descriptive text labels to perform well.

Related Skills

Explore More Data & Analytics Skills→
nielpattinnielpattin

visit-webpage

Retrieve web page content, converting HTML to markdown or downloading images.

Community
Intermediate
kaminocorpkaminocorp
1

clip

Generates image and text embeddings for zero-shot classification and cross-modal retrieval.

Official
Intermediate
ricablericable

clip

Classify images and score image-text similarity using CLIP embeddings.

Community
Intermediate
DoanNgocCuongDoanNgocCuong

llava

Process images with LLaVA for visual question answering and captioning.

Community
Intermediate