qsv-performance

What problem does it solve?

qsv performance can be slow on large CSV datasets due to repeated scans and lack of contextual caching. This guide shows how to speed up workflows by using index files, a stats cache, and frequency caches to avoid repeated work.

Core Features & Use Cases

• Index Files: Create per-file indices (.csv.idx) to enable instant row counts and O(1) lookups for repeated queries.
• Stats Cache: Maintain .stats.csv and .stats.csv.data.jsonl to accelerate frequency, schema, and join planning.
• Polars Engine: Use Polars-enabled commands (sqlp, joinp, pivotp) for large or complex analyses.
• Rule: Always enable the accelerators for repeated operations on the same file to realize the best gains.

Quick Start

Run a workflow that enables indexing, activates the stats cache, and uses the frequency cache to speed up a typical qsv analysis.