Skills
.Work. You
Rest

duplicate_row_check

Official

Efficiently detect duplicate rows in structured data files.

Data & Analytics#structured data#data validation#duplicate detection#CSV#QC marking
Authorcas-bigdatalab
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill addresses the challenge of identifying and marking duplicate rows in structured data files, such as CSV, TSV, and Excel, without the need for manual review.

Core Features & Use Cases

  • Duplicate Row Detection: Identifies exact duplicates or specified column subsets within structured data files.
  • Marking First Occurrences: Retains the first occurrence of a duplicate and marks subsequent duplicates with a quality control (QC) mark.
  • Use Case: Ideal for data validation in data processing pipelines where maintaining the integrity of the first occurrence is crucial.

Quick Start

Run the duplicate_row_check skill to detect and mark duplicate rows in your data file 'data.csv' and output the results to 'checked.csv'.

Dependency Matrix

Required Modules

None required

Components

scripts

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: duplicate_row_check
Download link: https://github.com/cas-bigdatalab/piflow/archive/main.zip#duplicate-row-check

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.