skills.rest
DamarisDeng
DamarisDengCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

load-and-profile

Scan folders for CSV and XLSX files and generate profile.json and variable_types.json.

Updated Mar 8, 2026
Search Tags:#profiling#pandas#dataset#data-quality#profile#load
One-click install
npx skills add https://github.com/DamarisDeng/paper-writing-system --skill load-and-profile
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: load-and-profile
Source: https://github.com/DamarisDeng/paper-writing-system/tree/main/workflow/skills/load-and-profile
Command: npx skills add https://github.com/DamarisDeng/paper-writing-system --skill load-and-profile

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas.

What problem does it solve?

Load and profile datasets from a folder to generate structured outputs (profile.json and variable_types.json) for downstream pipeline stages, enabling rapid data understanding and quality assessment.

Core Features & Use Cases

  • Automatically discovers CSV and XLSX files within a data folder (recursively) and profiles them.
  • Infers semantic variable types for each column and compiles per-file profiling data for validation and planning.
  • Useful at the start of data pipelines to quickly understand shapes, cleanliness, and readiness of multiple files across a dataset.

Quick Start

Run the load-and-profile skill on a folder containing your data to generate profile.json and variable_types.json in the output directory.

Frequently Asked Questions about load-and-profile

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I profile multiple CSV and XLSX files in a dataset folder?▼

Dataset profiling automates scanning a folder recursively to discover CSV and XLSX files, generating structured profile.json and variable_types.json outputs for rapid data quality assessment and pipeline ingestion.

What is the best way to automate data quality checks across multiple files?▼

Automated data quality checks infer semantic variable types for each column and compile per-file profiling data, enabling validation and planning for multiple files across a dataset folder.

Do I need pandas to generate a profile.json from my datasets?▼

Yes, pandas is required to load and profile datasets, infer semantic variable types, and compile structured profiling outputs for downstream pipeline stages and exploratory analyses.

Can I use automated profiling for start-of-pipeline data ingestion tasks?▼

Yes, automated profiling is applicable to start-of-pipeline data ingestion tasks, allowing you to quickly understand shapes, cleanliness, and readiness of multiple files across a dataset.

What file formats does automated dataset profiling support for scanning?▼

Automated dataset profiling supports scanning folders recursively to discover and profile CSV and XLSX files, generating structured outputs for downstream pipeline stages and exploratory analyses.

How does dataset profiling infer variable types for each column?▼

Dataset profiling infers semantic variable types for each column while compiling per-file profiling data, outputting a variable_types.json file to validate and plan downstream data workflows.

Related Skills

Explore More Data & Analytics Skills→
cas-bigdatalabcas-bigdatalab
541

table_collector

Extract structured data from CSV, TSV, XLS, and XLSX files into JSONL, JSON, or CSV.

Official
Intermediate
DamarisDengDamarisDeng

generate-research-questions

Generate ranked PICO/PECO research questions from profiled dataset JSON files.

Community
Advanced
AubreyHanAubreyHan

data-autocleaning

Clean and transform BigQuery and Google Cloud Storage data with Dataplex profiling.

Community
Advanced
WynterJonesWynterJones
5

data-analyst

Analyze datasets to produce statistical summaries and data-driven reports.

Community
Intermediate