skills.rest
aws-samples
aws-samplesOFFICIAL
Explore all skills by this author →
WebsiteGitHub

data-structure-checker

Detects and fixes common tabular file issues, returning a clean DataFrame for analysis.

48|12|Updated Nov 12, 2025
Search Tags:#pandas#data-cleaning#data-preparation#unicode-path#data-type-inference#encoding-detection#multi-level-headers
One-click install
npx skills add https://github.com/aws-samples/sample-deep-insight --skill data-structure-checker
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-structure-checker
Source: https://github.com/aws-samples/sample-deep-insight/tree/main/self-hosted/skills/data-structure-checker
Command: npx skills add https://github.com/aws-samples/sample-deep-insight --skill data-structure-checker

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, pandas, and includes scripts (resource) components.

What problem does it solve?

This skill automatically detects and fixes common data file issues when reading any tabular data file (Excel, CSV, Parquet, ODS). It returns a clean DataFrame ready for analysis with zero user intervention.

Core Features & Use Cases

  • Auto-fixes:
    • Multi-level/hierarchical headers (flattens with a separator)
    • Encoding issues (utf-8, cp949, euc-kr, etc.)
    • Empty rows and columns
    • Data type inference and conversion
    • Duplicate column names
    • Unicode path issues (Korean/CJK filenames)
  • Use Case: Quickly turn messy data files into analysis-ready DataFrames across formats (Excel, CSV, Parquet, ODS).

Quick Start

To read any tabular data file with automatic issue detection and fixing, run the checker script: uv run python skills/data-structure-checker/scripts/checker.py data.xlsx

Frequently Asked Questions about data-structure-checker

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I clean up messy Excel and CSV files with encoding issues and multi-level headers?▼

Data-structure-checker automatically detects and fixes common tabular data problems—encoding mismatches, hierarchical headers, empty rows/columns, and data type mismatches—across Excel, CSV, TSV, ODS, and Parquet formats, returning a clean pandas DataFrame ready for analysis.

Can I handle Unicode path issues like Korean or CJK filenames when reading data files?▼

Yes. The tool normalizes Unicode paths including Korean and CJK characters, enabling reliable reads of data files with non-ASCII filenames and preventing encoding-related failures during file access.

What's the best way to automatically infer and fix data types in a messy dataset?▼

Data-structure-checker performs automatic data-type inference and conversion, detecting type mismatches in your tabular data and converting columns to the correct types without manual intervention.

How do I flatten multi-level headers in pandas without manual column manipulation?▼

The tool detects hierarchical headers and automatically flattens them using a configurable separator, converting complex nested structures into simple, analysis-ready column names in a single step.

Does this handle duplicate column names and empty rows in large data files?▼

Yes. It removes empty rows and columns, renames duplicate column names automatically, and processes all supported formats—Excel, CSV, Parquet, ODS—at scale.

Related Skills

Explore More Data & Analytics Skills→
datheredathere
3.7k

data-convert

Convert tabular data files between CSV, TSV, Excel, JSONL, and Parquet formats.

Official
Advanced
Kgan01Kgan01
6

data-analysis

Analyze datasets with Pandas to produce descriptive statistics, visualizations, and insights reports.

Community
Advanced
jkitchinjkitchin
34

python-plotting

Select and apply Matplotlib, Seaborn, or Plotly for Python visualizations.

Community
Intermediate
Adelie-SquadAdelie-Squad
17

data-engineer

Clean and transform raw datasets into analysis-ready tables with pandas pipelines.

Official
Advanced