csv-pipeline

You can clean and transform messy CSV data by applying filtering, sorting, deduplication, and validation using Python 3 and standard command-line tools without requiring external dependencies.

Yes, you can convert between JSON Lines and CSV formats using Python's standard library. The process handles format conversion for tabular records without needing additional packages installed.

Join and aggregate large TSV datasets using streaming support to process massive files. The operation joins records on shared identifiers, groups rows, and calculates aggregates efficiently.

Generate summary reports from tabular data by grouping rows, calculating aggregates, and exporting the results as a Markdown report. This summarizes metrics like revenue by category for review.

Yes, csv-pipeline works with JSON and JSON Lines files for data validation and deduplication. It processes tabular records to filter, clean, and remove duplicates across these formats.

Processing large CSV files requires streaming support to avoid memory exhaustion. Streaming reads and transforms records sequentially, enabling filtering and aggregation on datasets too large for memory.