flagged_words_filter
OfficialFilter text based on the presence of flagged words above a specified threshold.
Content & Communication#data cleaning#text analysis#content moderation#text filtering#sensitive word detection
Authorcas-bigdatalab
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill is designed to address the need for filtering text that contains a proportion of flagged words above a specified threshold.
Core Features & Use Cases
- Text Filtering: Automatically remove samples from text that contain a high percentage of flagged words.
- Multilingual Support: Supports multiple languages for versatile usage.
- Use Case: It can be utilized in content moderation, data cleaning, and any scenario where the presence of sensitive words needs to be minimized.
Quick Start
Run the flagged_words_filter skill with the input file 'sample_input.json' and output to 'filtered_output.json'.
Dependency Matrix
Required Modules
data_juicer
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: flagged_words_filter Download link: https://github.com/cas-bigdatalab/piflow/archive/main.zip#flagged-words-filter Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.