data-packaging

What problem does it solve?

Package reduced time-resolved neutron reflectometry (tNR) outputs together with EIS timing metadata into a standardized Parquet dataset so data can be ingested into an Apache Iceberg lakehouse and used directly for downstream analysis.

Core Features & Use Cases

• Combine reduced reflectivity and EIS metadata: Merges per-run reflectivity files with interval JSON and the reduction template into a single analysis-ready Parquet table.
• Produce both data and metadata artifacts: Emits a main tnr_data.parquet with per-file reflectivity arrays and a tnr_data_metadata.parquet containing experiment-level metadata, reduction template, and full intervals JSON.
• Validation and safe packaging: Supports input validation mode to check required files and formats without creating outputs, and documents column mappings and Q/R ranges for reproducibility.
• Use Case: Prepare a full experiment's reduced tNR results for large-scale analytics and storage in an Iceberg lakehouse, enabling downstream model fitting, visualization, or ML workflows.

Quick Start

Package reduced tNR results using intervals.json, results/tnr, and template.xml into results/tnr/tnr_dataset.parquet.