skills.rest
alongor666
alongor666COMMUNITY
Explore all skills by this author →
GitHub

data-cleaning-standards

Clean and standardize vehicle insurance data in CSV/Excel inputs.

1|Updated Nov 4, 2025
Search Tags:#data quality#standardization#deduplication#data cleaning#Pandas#insurance data#missing values
One-click install
npx skills add https://github.com/alongor666/daylyreport --skill data-cleaning-standards
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-cleaning-standards
Source: https://github.com/alongor666/daylyreport/tree/main/.claude/skills/data-cleaning-standards
Command: npx skills add https://github.com/alongor666/daylyreport --skill data-cleaning-standards

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas.

What problem does it solve?

This Skill addresses the common challenges of messy, inconsistent vehicle insurance data, such as missing values, duplicate records, and non-standardized formats. It ensures data quality and reliability for accurate analysis.

Core Features & Use Cases

  • Missing Value Handling: Implements strategies for filling or removing nulls based on field type (e.g., fill '手续费含税' with 0, delete rows for '签单/批改保费').
  • Duplicate Removal: Deduplicates records using a composite key (保单号 + 投保确认时间), keeping the most recent entry.
  • Format Standardization: Converts dates to datetime64[ns], numerics to float64, and strings (stripping whitespace, mapping 'Y'/'N' to '是'/'否').
  • Use Case: A user receives a monthly Excel report with raw vehicle insurance data. This skill can automatically clean, standardize, and deduplicate the data, making it ready for immediate analysis and reporting.

Quick Start

Clean and standardize the attached vehicle insurance data file 'monthly_report.csv'.

Frequently Asked Questions about data-cleaning-standards

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I clean and standardize vehicle insurance data with missing values and duplicates?▼

Data cleaning standardizes insurance records by handling missing values through field-specific strategies, removing duplicates using composite keys, and normalizing dates, numerics, and text formats. This prepares raw CSV or Excel data for reliable analysis by ensuring consistency across all records.

What's the best way to deduplicate insurance records when policy numbers repeat?▼

Composite-key deduplication uses policy number plus confirmation timestamp to identify and remove duplicate records, retaining the most recent entry. This approach preserves data integrity while eliminating redundant policy entries in your dataset.

Can I use Pandas to standardize date formats and numeric types in insurance data?▼

Yes, Pandas converts dates to datetime64[ns] and numerics to float64, plus strips whitespace from strings and maps text values like 'Y'/'N' to standardized forms. This ensures consistent, analysis-ready formatting across all fields.

How do I handle missing values differently based on field type in insurance datasets?▼

Field-specific missing-value strategies fill or remove nulls based on column purpose—for example, filling fee fields with zero or deleting rows where premium data is absent. This preserves meaningful data while removing incomplete records.

What input formats does this data cleaning workflow support?▼

The standardization workflow processes CSV and Excel files containing vehicle insurance records. It accepts standard tabular inputs and outputs a cleaned, deduplicated dataset ready for downstream analysis.

Related Skills

Explore More Data & Analytics Skills→
alongor666alongor666
1

field-validation

Validate vehicle insurance CSV/Excel fields with a 4-layer pipeline and quality report.

Community
Intermediate
MichaelYang-lyxMichaelYang-lyx
110

excel-statistical-viz-large-file

Analyze large Excel datasets with pandas, numpy, matplotlib, and scipy.

Community
Advanced
LeandroBenjaminLLeandroBenjaminL

data-profiling

Profile datasets and validate schemas using pandas, ydata-profiling, and pandera.

Community
Intermediate
dawsonblockdawsonblock

nemo-curator

Automates GPU-accelerated data curation for LLM training datasets.

Community
Advanced