mad-dynamic-batching
CommunityRobustly batch variable tokens with MAD
Data & Analytics#token budgeting#robust statistics#mad#dynamic batching#quantile partitioning#machine learning training#outlier filtering
Authorthistleknot
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill prepares efficient ML training batches from variable-length items when the token-length distribution has outliers, ensuring batching is both robust and balanced.
Core Features & Use Cases
- MAD-gated center filtering: Computes median and MAD, converts MAD to a sigma-equivalent scale, and filters items within a 95% confidence band around the median.
- Equal-sized quantile partitions: Splits retained items into 4 equal quantile partitions per side of the median (center-outward layout), preventing skewed partition sizes.
- Anchor-pack batching: Packs each partition with longest-first “anchor” sequences to meet a per-batch token budget efficiently.
Quick Start
Use the skill to batch your dataset by token_count using MAD gating, equal quantile partitions, and anchor-pack packing under your target token budget.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: mad-dynamic-batching Download link: https://github.com/thistleknot/skills/archive/main.zip#mad-dynamic-batching Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.