skills.rest
sunLeee
sunLeeeCOMMUNITY
Explore all skills by this author →
GitHub

convention-data-handling

Enforce standardized data-handling conventions for missing data, outliers, vectorization, and scaling.

Updated Mar 19, 2026
Search Tags:#data#conventions#scaling#vectorization#outliers#missing-data
One-click install
npx skills add https://github.com/sunLeee/optimization --skill convention-data-handling
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: convention-data-handling
Source: https://github.com/sunLeee/optimization/tree/main/.claude/skills/reference/convention/convention-data-handling
Command: npx skills add https://github.com/sunLeee/optimization --skill convention-data-handling

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data engineers and analysts often struggle with inconsistent handling of missing data, outliers, vectorization, and scaling across pipelines, which can cause unreliable results and inefficient processing. This skill provides a standardized, actionable guide to data-handling conventions that improve quality, reproducibility, and performance.

Core Features & Use Cases

  • Missing data strategies: guidelines for detection, imputation, and validation to ensure data completeness.
  • Outlier handling: detection using robust methods and practical remediation like capping or transformation.
  • Vectorization best practices: promote Pandas/Numpy vectorized operations over explicit Python loops for speed and memory efficiency.
  • Scaling for large datasets: memory-aware data types and chunked processing to enable scalable analytics.
  • Use Case: apply conventions to a customer analytics dataset to improve model reliability and processing times.

Quick Start

Apply these conventions to a new dataset by identifying missing values, selecting an imputation strategy, detecting outliers, choosing vectorized operations, and selecting memory-efficient data types.

Frequently Asked Questions about convention-data-handling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is the best way to handle missing data and outliers in an analytics pipeline?▼

Standardized data handling enforces guidelines for detecting and imputing missing data, remediating outliers through capping, and applying vectorized operations to improve pipeline reproducibility and processing speed.

How do I standardize data scaling for large datasets without running out of memory?▼

Standardize data scaling for large datasets by applying memory-aware data types and chunked processing conventions, enabling scalable analytics while maintaining memory efficiency during vectorized operations.

How do I vectorize Pandas and Numpy operations to improve data processing performance?▼

Vectorize Pandas and Numpy operations by replacing explicit Python loops with standardized vectorized processing conventions, achieving deterministic execution, speed improvements, and memory efficiency in data pipelines.

Does applying data handling conventions work for customer analytics datasets?▼

Applying data handling conventions works for customer analytics datasets by identifying missing values, selecting imputation strategies, detecting outliers, and choosing memory-efficient data types to improve model reliability.

When should I use robust outlier detection methods in a data pipeline?▼

Use robust outlier detection methods in data pipelines when inconsistent data handling threatens reliability, applying standardized remediation techniques like capping or transformation to maintain data quality and reproducibility.

Related Skills

Explore More Data & Analytics Skills→
hung-phanhung-phan
1

feature-engineering

Transform raw datasets into machine learning features with scaling, encoding, and vectorization.

Community
Advanced
nksethnkseth

pandas-pro

Perform vectorized transformations and data validation on pandas DataFrames.

Community
Advanced
SnakeOSnakeO
9

refactor:pandas

Refactor Pandas row-wise operations into vectorized pipelines with PyArrow.

Community
Intermediate
erlebacherlebach

Data Cleaning Pipeline

Automate data cleaning workflows for missing values, outliers, and validation.

Community
Advanced