sn-da-large-file-analysis

What problem does it solve?

This Skill addresses the challenge of analyzing very large Excel datasets that exceed standard processing capabilities, enabling high-speed, memory-efficient workflows.

Core Features & Use Cases

• Stream Inspection: Examine large Excel files' sheet structures and row counts without loading entire data into memory.
• Streaming Read and Conversion: Convert huge Excel files into Parquet format incrementally, avoiding memory overload.
• High-Performance Analysis: Perform data aggregation, filtering, and export tasks on datasets with millions of rows efficiently.
• Use Case Example: Quickly analyze a 1 million row transaction log to filter high-value transactions and generate export files without crashing or excessive delays.

Quick Start

Use the analysis skill to read a large Excel file and produce a summarized report or export filtered results with minimal memory usage.