ingest

Import datasets into a Mycelium project with metadata, provenance, and manifest registration.

68|2|Updated Feb 25, 2026
One-click install
npx skills add https://github.com/arjunrajlaboratory/mycelium --skill ingest-arjunrajlaboratory
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ingest
Source: https://github.com/arjunrajlaboratory/mycelium/tree/main/skills/ingest
Command: npx skills add https://github.com/arjunrajlaboratory/mycelium --skill ingest-arjunrajlaboratory

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? New datasets arriving from collaborators or facilities often get dropped into projects without documentation, making them impossible to interpret or reuse later. This Skill registers each dataset with structured metadata, provenance, and validation so it becomes a durable, documented part of the analytical project. ## Core Features & Use Cases - Structured data placement: Organizes raw data into per-dataset directories under data/raw/ with UPPER_SNAKE_CASE documentation files, treating raw data as immutable. - Metadata generation: Creates schema.yaml, provenance.md, and summary_stats.md from templates, capturing source, acquisition date, column descriptions, known issues, and access restrictions. - Manifest and decision logging: Updates data/DATA_MANIFEST.md with a new entry and logs ingestion decisions and learnings to the .living/ directory. - Domain-aware validation: Consults installed domain conventions (e.g., bioinformatics, image-analysis) for format-specific QC requirements. - Use Case: A sequencing facility delivers FASTQ files for a new experiment. Use this Skill to place them in data/raw/, generate provenance and schema metadata, apply bioinformatics QC conventions, and register the dataset in the manifest. ## Quick Start Use the ingest skill to add these FASTQ files from the sequencing facility to my project with full metadata and provenance.

Frequently Asked Questions about ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new dataset to a Mycelium project?

Place the files in a new subdirectory under data/raw/, generate schema.yaml, provenance.md, and summary_stats.md in data/metadata/, then add an entry to data/DATA_MANIFEST.md. The ingest skill walks through each of these steps using bundled templates.

What file formats can I ingest into a Mycelium project?

The ingest workflow supports common analytical formats including CSV, FASTQ, XLSX, TIFF, JSON, FCS, TSV, and image files. Domain conventions can add format-specific validation, such as QC metrics for sequencing data.

How does the ingest skill handle large data files?

Files too large for git are handled with a .gitignore entry plus download documentation so the dataset remains reproducible. The skill plans this during ingestion when it detects large files.

When should I not use the ingest skill?

Do not use it for analyzing existing data, listing datasets, deleting data, downloading from URLs, or editing code. It is strictly for adding or importing new files and datasets with metadata and provenance.

What metadata is required when ingesting a dataset?

Required fields include source, date acquired, schema or column descriptions, known issues, and access restrictions. These are captured in schema.yaml, provenance.md, and summary_stats.md generated from the core templates.