dataset

Name: dataset
Availability: InStock
Author: Nebius

Ingest, validate, and curate sensor data into versioned datasets.

17|8|Updated Apr 7, 2026

Search Tags:#data lineage #s3 #dataset #data curation #sensor data #physical ai

Ready to run

npx skills add https://github.com/nebius/nebius-physical-ai --skill dataset-nebius

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dataset
Source: https://github.com/nebius/nebius-physical-ai/tree/main/skills/tools/dataset
Command: npx skills add https://github.com/nebius/nebius-physical-ai --skill dataset-nebius

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill solves the fragmentation of sensor data management by providing a unified, versioned, and queryable dataset-of-record that integrates ingestion, validation, and curation into a single workflow.

Core Features & Use Cases

Unified Data Pipeline: Streamlines the ingestion, validation, and curation of raw sensor data into versioned manifests.
Quality-Driven Curation: Enables filtering and slicing of datasets based on specific events, locations, and quality metrics.
Use Case: A robotics team can ingest raw sensor logs from S3, validate them against schema requirements, and curate a high-quality subset for model training, all while maintaining full lineage provenance.

Quick Start

Use the dataset skill to ingest raw sensor data from the specified S3 input path and register a versioned manifest at the output path.

Frequently Asked Questions about dataset

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I manage sensor data ingestion and curation for physical AI workflows?▼

Sensor data ingestion and curation for physical AI is managed by transforming raw logs from S3 into versioned, queryable records. This process integrates validation and lineage tracking to produce quality-gated datasets suitable for model training.

How do I maintain data lineage across versioned datasets stored in S3?▼

Maintaining data lineage across versioned datasets in S3 requires a unified pipeline that registers manifests and tracks provenance from raw ingestion to curated output. This ensures full traceability of every record used in model training.

Can I use LanceDB to index and query curated sensor datasets?▼

LanceDB can optionally be integrated to index and support high-performance retrieval of curated sensor datasets. This works alongside S3 storage to enable fast querying of quality-gated, versioned records.

What is the best way to filter raw sensor logs for model training based on quality metrics?▼

Filtering raw sensor logs for model training is best handled through quality-driven curation that slices datasets based on specific events, locations, and quality metrics. This validates logs against schema requirements before creating a training subset.

Does this approach to dataset versioning work with existing physical AI pipelines?▼

This dataset versioning approach works within physical AI pipelines by operating directly on raw sensor logs stored in S3. It requires no additional dependencies, using optional LanceDB indexing to support retrieval and quality-gated versioning.

When do I need schema validation for production sensor data?▼

Schema validation for production sensor data is needed during the ingestion phase to ensure raw logs meet specific structural requirements before curation. This prevents invalid records from entering the versioned dataset-of-record.