special_characters_filter
OfficialEfficiently filter text samples based on special character ratios.
Data & Analytics#data cleaning#data preprocessing#text filtering#special characters#text quality control
Authorcas-bigdatalab
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill helps users filter text samples based on the ratio of special characters, ensuring that the content meets specific criteria.
Core Features & Use Cases
- Special Character Filtering: Filters text samples based on the ratio of special characters to maintain the quality of the text.
- Use Case: Ideal for data preprocessing in text analysis, such as removing spam messages or cleaning up input data before further processing.
Quick Start
Use the special_characters_filter skill to filter text samples with a special character ratio between 0.0 and 0.25 from the input file 'text_samples.jsonl'.
Dependency Matrix
Required Modules
data_juicer
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: special_characters_filter Download link: https://github.com/cas-bigdatalab/piflow/archive/main.zip#special-characters-filter Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.