Ingest, validate, and curate sensor data into versioned datasets.
npx skills add https://github.com/nebius/nebius-physical-ai --skill dataset-nebius
Please help me install this Agent Skill. Skill: dataset Source: https://github.com/nebius/nebius-physical-ai/tree/main/skills/tools/dataset Command: npx skills add https://github.com/nebius/nebius-physical-ai --skill dataset-nebius
This skill solves the fragmentation of sensor data management by providing a unified, versioned, and queryable dataset-of-record that integrates ingestion, validation, and curation into a single workflow.
Use the dataset skill to ingest raw sensor data from the specified S3 input path and register a versioned manifest at the output path.
High-intent search queries and answers about installing and using this skill.
Sensor data ingestion and curation for physical AI is managed by transforming raw logs from S3 into versioned, queryable records. This process integrates validation and lineage tracking to produce quality-gated datasets suitable for model training.
Maintaining data lineage across versioned datasets in S3 requires a unified pipeline that registers manifests and tracks provenance from raw ingestion to curated output. This ensures full traceability of every record used in model training.
LanceDB can optionally be integrated to index and support high-performance retrieval of curated sensor datasets. This works alongside S3 storage to enable fast querying of quality-gated, versioned records.
Filtering raw sensor logs for model training is best handled through quality-driven curation that slices datasets based on specific events, locations, and quality metrics. This validates logs against schema requirements before creating a training subset.
This dataset versioning approach works within physical AI pipelines by operating directly on raw sensor logs stored in S3. It requires no additional dependencies, using optional LanceDB indexing to support retrieval and quality-gated versioning.
Schema validation for production sensor data is needed during the ingestion phase to ensure raw logs meet specific structural requirements before curation. This prevents invalid records from entering the versioned dataset-of-record.
AradotsoProvision AWS S3 buckets and EC2 instances with Terraform for data engineering.
jluo41Package trained AI models into versioned assets with training results.
hai-seriesManage collections, vectors, and similarity search across LanceDB, pgvector, and Qdrant.
PromakerDevDiagnoses a specified application and outputs a structured report.
