Convert local datasets from files or directories to JSONL format.
npx skills add https://github.com/cas-bigdatalab/piflow --skill local-formatter
Please help me install this Agent Skill. Skill: local_formatter Source: https://github.com/cas-bigdatalab/piflow/tree/main/workspace/skills/local_formatter Command: npx skills add https://github.com/cas-bigdatalab/piflow --skill local-formatter
The local_formatter Skill unit simplifies the process of loading and formatting local datasets, enabling users to quickly prepare data for analysis.
Run the local_formatter Skill with the following command: python scripts/run_local_formatter.py --input_path /path/to/input --output_path /path/to/output
High-intent search queries and answers about installing and using this skill.
Converting local datasets to JSONL format involves loading files or directories and standardizing their structure. This process outputs uniform JSONL data ready for consumption by downstream data analysis tools and systems.
Standardizing local data files into a uniform format is best achieved by converting them into JSONL. This ensures consistent data structure across multiple file types, making the data immediately ready for further analysis.
Yes, you need py-data-juicer to process local datasets and convert them into JSONL format. It acts as the underlying dependency required to load various file types and standardize the data output.
Yes, you can convert an entire directory of local data files into a single JSONL output. By specifying the directory as the input path, the formatting process loads all compatible files and outputs them in a consistent JSONL format.
To run a script to convert local data to JSONL format, execute the formatting script via Python and provide the input and output paths as arguments. This processes the local files and saves the standardized JSONL dataset.
EurecaMomentClean and preprocess datasets using Data-Juicer tools.
synaptiaiMerge multiple sources into one state with configurable conflict resolution strategies.
patricio0312revConvert documents into chunked JSONL datasets for embeddings.
89jobrienGenerate Python code to convert files between document, data, and image formats.
