jn

Streaming ETL pipelines convert CSV to JSON by reading CSV files with jn cat, transforming rows into NDJSON format, then writing output as JSON. This approach handles large datasets without loading everything into memory, enabling efficient format conversion across CSV, JSON, Excel, and YAML sources.

Yes, jn filter applies jq-like expressions to NDJSON records directly in Unix pipes, enabling transformation and filtering without database infrastructure. Results stream to stdout for composition with other commands or write operations using jn put.

Build ETL workflows by chaining jn cat to read sources, jn filter to transform with expressions, and jn put to write outputs. Unix pipe composition lets you handle CSV, JSON, Excel, and YAML in a single streaming pipeline with NDJSON as the universal interchange format.

The -s slurp option enables in-memory aggregation when you need batch-style operations across entire datasets. Format forcing like ~json controls output serialization; use slurp mode for aggregations that require complete dataset access before transformation.

NDJSON pipelines support filtering at scale through streaming processing; jn filter reads and processes records one at a time without holding full datasets in memory. Streaming composition with Unix pipes lets you chain filters, transformations, and outputs efficiently on large data volumes.

ETL workflows support conversion between CSV, JSON, Excel, and YAML formats using jn cat for input and jn put for output. NDJSON serves as the universal intermediate format; all sources convert to NDJSON for processing, then output to your target format.