skills.rest
tamagusko
tamaguskoCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

clip

Performs zero-shot image classification and image-text similarity using OpenAI's CLIP model.

Updated Oct 7, 2022
Search Tags:#embeddings#image search#pytorch#content moderation#clip#vision-language#zero-shot classification
One-click install
npx skills add https://github.com/tamagusko/linux-cfg --skill clip-tamagusko
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/tamagusko/linux-cfg/tree/main/dotfiles/claude/skills/clip
Command: npx skills add https://github.com/tamagusko/linux-cfg --skill clip-tamagusko

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? Classifying images or searching image collections normally requires training a custom model on labeled data. CLIP removes that requirement by matching images against natural-language descriptions, enabling zero-shot classification, semantic image search, and content moderation without any fine-tuning. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text categories with no training data, using models from RN50 up to ViT-L/14. - Semantic Image Search & Retrieval: Encode images and text into a shared embedding space for cross-modal search, and integrate embeddings with vector databases like Chroma or FAISS. - Content Moderation & Deduplication: Score images against safety categories or detect near-duplicate images via cosine similarity. - Use Case: Index a folder of product photos, then query it with "a red sneaker on white background" to instantly retrieve the most relevant images ranked by similarity. ## Quick Start Use the clip skill to classify photo.jpg against the labels dog, cat, bird, and car and show the probability for each.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then call model(image, text) and apply softmax to the logits to get per-label probabilities. No training data is required.

How to build semantic image search with CLIP embeddings?▼

Encode every image with model.encode_image, normalize the embeddings, then encode a text query and compute cosine similarity against all image embeddings. Store embeddings in Chroma or FAISS for fast retrieval at scale.

Which CLIP model should I use: RN50, ViT-B/32, or ViT-L/14?▼

ViT-B/32 is the recommended default, balancing speed and quality at 151M parameters. RN50 is fastest, while ViT-L/14 gives the best quality but is the slowest at 428M parameters.

Does CLIP work on CPU or does it require a GPU?▼

CLIP runs on CPU but a GPU is 10-50x faster. Image encoding takes roughly 200ms on CPU versus about 20ms on a V100 GPU, so GPU is recommended for batch processing.

What are the limitations of CLIP for image understanding?▼

CLIP works best for broad categories, not fine-grained tasks. It produces no bounding boxes, has weak spatial and counting understanding, needs descriptive text labels, and can reflect biases from its web training data.

Related Skills

Explore More Data & Analytics Skills→
nielpattinnielpattin

visit-webpage

Retrieve web page content, converting HTML to markdown or downloading images.

Community
Intermediate
kaminocorpkaminocorp
1

clip

Generates image and text embeddings for zero-shot classification and cross-modal retrieval.

Official
Intermediate
curiositechcuriositech
181

clip-aware-embeddings

Guide CLIP image-text matching with decision trees and alternative model recommendations.

Official
Advanced
MLT-OSSMLT-OSS
14

llava

Run vision-language inference and fine-tuning with LLaVA models for image understanding.

Official
Intermediate