skills.rest
MedTiLab
MedTiLabCOMMUNITY
Explore all skills by this author →
GitHub

clip

Performs zero-shot image classification and image-text retrieval using CLIP embeddings.

5|2|Updated Aug 10, 2026
Search Tags:#embeddings#image search#content moderation#zero-shot#CLIP#vision-language#image classification
One-click install
npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill clip-medtilab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/MedTiLab/Auto-meta-analysis/tree/main/skills/multimodal/clip
Command: npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill clip-medtilab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? Classifying images or searching image collections normally requires labeled training data and model fine-tuning. This Skill lets you classify images, match images to text, and build semantic image search using OpenAI's CLIP model with zero training data. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text-defined categories without any training, using models like ViT-B/32. - Semantic Image Search & Cross-Modal Retrieval: Encode images and text into a shared embedding space to search images with natural language queries or find captions matching an image. - Content Moderation & Deduplication: Score images against safety categories (NSFW, violence) or detect near-duplicate images via cosine similarity of embeddings. - Use Case: Index a folder of product photos by computing CLIP embeddings, then retrieve the most relevant images for the query "a sunset over the ocean" ranked by similarity score. ## Quick Start Use the clip skill to classify the attached image 'photo.jpg' against the labels dog, cat, bird, and car.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then pass both through the model to get logits. Apply softmax to the logits to obtain per-label probabilities without any training data.

How to build semantic image search with CLIP embeddings?▼

Encode each image with model.encode_image, normalize the embeddings, and store them. Encode a text query the same way, then rank images by cosine similarity between the text embedding and stored image embeddings.

Which CLIP model should I use: RN50, ViT-B/32, or ViT-L/14?▼

ViT-B/32 is the recommended default, balancing speed and quality at 151M parameters. RN50 is fastest, while ViT-L/14 gives the best quality but is the slowest at 428M parameters.

Does CLIP work on CPU or does it require a GPU?▼

CLIP runs on CPU, with image encoding around 200ms and text encoding around 50ms. A GPU such as a V100 is recommended since it is roughly 10-50x faster, about 20ms per image.

What are the limitations of CLIP for image understanding?▼

CLIP is weak at fine-grained tasks, spatial reasoning, counting, and object localization since it produces whole-image embeddings with no bounding boxes. It also requires descriptive text labels and may carry biases from its web training data.

Related Skills

Explore More Data & Analytics Skills→
steelan9199steelan9199
9

webp-to-jpg

Convert WebP images to JPG with adjustable quality using Pillow.

Community
Basic
tawf-labstawf-labs
1

langchain-rag

Build retrieval-augmented generation workflows that ground LLM answers in external documents.

Official
Intermediate
blueskies1818blueskies1818
1

segment-anything-model

Segment objects in images using the Segment Anything Model.

Community
Intermediate
MLT-OSSMLT-OSS
14

llava

Run vision-language inference and fine-tuning with LLaVA models for image understanding.

Official
Intermediate