skills.rest
ricable
ricableCOMMUNITY
Explore all skills by this author →
GitHub

ray-data

Streams distributed, batched ML data preprocessing across clusters for PyTorch and TensorFlow workflows.

Updated Feb 15, 2026
Search Tags:#machine learning#etl#ray data#gpu acceleration#batch inference#streaming execution#distributed processing
One-click install
npx skills add https://github.com/ricable/mcai --skill ray-data-ricable
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ray-data
Source: https://github.com/ricable/mcai/tree/main/.agents/skills/ray-data
Command: npx skills add https://github.com/ricable/mcai --skill ray-data-ricable

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Ray Data removes the bottleneck of processing large machine learning datasets on a single machine by enabling distributed, streaming data pipelines that can grow from a laptop to a cluster.

Core Features & Use Cases

  • Large-Scale Preprocessing: Clean, transform, filter, and repartition massive datasets efficiently with batch-oriented operations.
  • Batch Inference and Training Pipelines: Feed preprocessed data directly into Ray Train, PyTorch, or TensorFlow workflows.
  • Multi-Modal Data Handling: Work with Parquet, CSV, JSON, images, and other ML data types for end-to-end ETL and inference jobs.
  • Use Case: A team preparing 100GB of training data can stream records from cloud storage, apply GPU-accelerated image transforms, and write the processed output back to Parquet for model training.

Quick Start

Use Ray Data to load a large dataset, apply batch transformations, and write the processed results back to storage.

Frequently Asked Questions about ray-data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I process large machine learning datasets without loading them into memory?▼

You can process large machine learning datasets without loading them into memory by using distributed, streaming data pipelines. This approach enables batch-oriented operations to clean, transform, and filter records incrementally across a cluster.

How do I run distributed preprocessing on multi-modal data like images and audio?▼

Distributed preprocessing on multi-modal data like images and audio is handled through streaming execution and GPU-accelerated batch transforms. The system supports multi-modal ingestion across Parquet, CSV, JSON, images, audio, and video formats.

Can I feed preprocessed data directly into Ray Train, PyTorch, or TensorFlow workflows?▼

Yes, preprocessed data can be fed directly into Ray Train, PyTorch, or TensorFlow workflows. This integration supports batch inference and training pipelines by efficiently passing transformed records into your ML frameworks.

What is the best way to scale ML data prep from a laptop to a cluster?▼

The best way to scale ML data prep from a laptop to a cluster is using distributed ETL pipelines. This removes single-machine bottlenecks by enabling streaming execution that seamlessly grows from local execution to cluster-scale processing.

Does distributed batch inference work with GPU acceleration?▼

Yes, distributed batch inference works with GPU acceleration. By applying GPU-accelerated batch transforms to streaming records from cloud storage, you can efficiently process large volumes of data before writing the output back to Parquet.

Related Skills

Explore More Data & Analytics Skills→
qcmuuqcmuu

ray-data

Scale ML data preprocessing and batch inference with Ray Data across distributed workers.

Community
Advanced
hhhi21ghhhi21g

ray-data

Process large datasets for ML with distributed computing across CPU/GPU environments.

Community
Advanced
Tnemo65Tnemo65

ray-data

Scale distributed data processing for ML workloads with streaming and batch pipelines.

Community
Advanced
Clay-HHKClay-HHK
2

ray-data

Process large-scale ML datasets across clusters for distributed preprocessing and batch inference.

Community
Advanced