Classify images and compute image-text similarity using OpenAI's CLIP model.
npx skills add https://github.com/xu1713/openhorse --skill clip-xu1713
Please help me install this Agent Skill. Skill: clip Source: https://github.com/xu1713/openhorse/tree/main/openhorse/openhorse/optional-skills/mlops/clip Command: npx skills add https://github.com/xu1713/openhorse --skill clip-xu1713
High-intent search queries and answers about installing and using this skill.
Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then call model(image, text) to get logits. Apply softmax to get probabilities for each label without any training.
ViT-B/32 is recommended for most cases, balancing speed and quality with 151M parameters. RN50 is fastest, while ViT-L/14 gives the best quality but is the slowest at 428M parameters.
Yes, encode images into normalized embeddings, then encode a text query and compute cosine similarity to rank images. Embeddings can be stored in vector databases like Chroma or FAISS for efficient retrieval.
No, CLIP runs on CPU, but a GPU is 10-50x faster. Image encoding takes about 200ms on CPU versus 20ms on a V100 GPU, so GPU is recommended for batch processing.
CLIP works best for broad categories, not fine-grained tasks. It cannot produce bounding boxes, has weak spatial and counting understanding, requires descriptive text labels, and may carry biases from its web training data.
Darsh20009Remove backgrounds from images and output transparent PNGs.
tawf-labsBuild retrieval-augmented generation workflows that ground LLM answers in external documents.
robertbr123Segment objects in images using the Segment Anything Model with point, box, or mask prompts.
OpenLAIRBridge frozen vision encoders with large language models for multimodal reasoning.
