skills.rest
anhvth
anhvthCOMMUNITY
Explore all skills by this author →
GitHub

vision-utilities

Load and visualize image datasets for ML workflows with optional DALI acceleration.

9|Updated Aug 8, 2023
Search Tags:#vision#datasets#gpu-acceleration#image-loading#memory-mapped#notebook-visualization
One-click install
npx skills add https://github.com/anhvth/speedy_utils --skill vision-utilities
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vision-utilities
Source: https://github.com/anhvth/speedy_utils/tree/main/.github/skills/vision-utilities
Command: npx skills add https://github.com/anhvth/speedy_utils --skill vision-utilities

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Vision utilities simplify efficient image loading, memory-mapped datasets, and notebook visualization for computer vision workflows, reducing setup complexity and bench time.

Core Features & Use Cases

  • Fast Image Loading: GPU-accelerated with optional DALI, CPU fallback, batch processing, and image validation.
  • Memory-Mapped Datasets: ImageMmap and ImageMmapDynamic for fixed and variable-size images with multi-process safety and fast access.
  • Notebook Visualization: plot_images_notebook for auto-grid visualization of mixed inputs in notebooks.
  • Use cases include training pipelines that require quick data loading, large datasets, and quick sanity checks in notebooks.

Quick Start

Run a simple example to read images, build an mmap dataset, and visualize results in a notebook.

Frequently Asked Questions about vision-utilities

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I speed up image loading for large-scale vision datasets?▼

Memory-mapped datasets store fixed and variable-size images on disk for rapid access without loading everything into RAM. They ensure multi-process safety and fast retrieval during large-scale vision training workflows.

Can I visualize images directly in a Jupyter notebook during ML experiments?▼

You can visualize images directly in a Jupyter notebook using auto-grid plotting functions designed for mixed inputs. It provides immediate sanity checks with CPU fallbacks if GPU acceleration is unavailable.

Does this image loading utility require a GPU to function?▼

A GPU is not required as the utility provides CPU fallbacks for both image loading and notebook visualization. GPU acceleration via DALI is optional and used to further accelerate batch processing when available.

What's the best way to handle variable-size images in a memory-mapped dataset?▼

Handle variable-size images in a memory-mapped dataset using the ImageMmapDynamic format, which allows fast access and multi-process safety for fixed and variable dimensions without manual padding.

How do I validate images during batch processing in a training pipeline?▼

Validate images during batch processing by utilizing the fast image loading utility's built-in validation features, ensuring corrupted or incompatible files are caught before entering the training loop.

Related Skills

Explore More Data & Analytics Skills→
omosb1-sysomosb1-sys

hardware-optimizer

Optimize macOS hardware usage for Polars and DuckDB data processing.

Community
Advanced
KentoShimizuKentoShimizu
7

ml-training-optimization

Optimize ML training workflows for throughput, convergence and cost control.

Community
Intermediate
yo-stevenyo-steven

dali-dynamic-mode

Implement DALI dynamic mode data loading with GPU decode and PyTorch tensor conversion.

Community
Advanced
tykimostykimos

ssw-plugin:ssw-ml

Train deep learning models for solar flare prediction and image-to-image translation.

Community
Advanced