Process large tabular datasets out-of-core with lazy Vaex DataFrames.
npx skills add https://github.com/shushuzn/Rairos --skill vaex-shushuzn
Please help me install this Agent Skill. Skill: vaex Source: https://github.com/shushuzn/Rairos/tree/main/skills/vaex Command: npx skills add https://github.com/shushuzn/Rairos --skill vaex-shushuzn
Vaex helps you analyze and visualize tabular datasets that are too large to fit into memory by using lazy, out-of-core DataFrame operations. This avoids slow, memory-exhausting workflows when your CSV/Parquet/HDF5/Arrow data spans billions of rows.
Open your large dataset with Vaex (for example, using an HDF5/Arrow/Parquet file) so you can compute statistics and build plots without loading the full data into RAM.
High-intent search queries and answers about installing and using this skill.
Out-of-core processing analyzes large CSV or Parquet datasets that exceed available RAM by using lazy evaluation to load files instantly and compute aggregations only when results are needed, avoiding memory exhaustion.
Lazy evaluation for big data analytics delays computations until results are explicitly requested, allowing huge tabular files like HDF5 and Arrow to be opened instantly without loading the entire dataset into memory.
Virtual columns and efficient filters are created using expression-based computations that operate on-the-fly, allowing you to derive features and filter billions of rows without materializing the full data in memory.
Interactive visualizations and ML-ready features can be generated for datasets larger than memory by running fast aggregations and building plots directly within the lazy DataFrame framework.
Out-of-core big data analytics works directly with HDF5 and Arrow file formats to load huge files instantly, and can convert large CSVs to these faster storage formats for repeated workflows.
The best way to convert large CSV files to efficient storage formats for repeated data visualization is exporting them to HDF5 or Arrow, which enables instant loading and lazy processing for scalable exploration.
ovachieverProcess and analyze out-of-core datasets exceeding RAM with Vaex DataFrames.
JNZader-VaultBuild FastAPI AI/ML services with LangChain and ONNX model export.
mkurmanProcess out-of-core tabular datasets with Vaex's lazy DataFrames.
m0atCreate out-of-core DataFrames to browse and aggregate huge datasets efficiently with lazy evaluation and versatile IO formats.
