What problem does it solve?
ML datasets often arrive in a schema that does not match the format required by SageMaker training or evaluation jobs, forcing developers to hand-write brittle conversion code. This Skill guides an agent through an 11-step workflow to generate, test, and execute a validated dataset transformation function.
Core Features & Use Cases
- Format Conversion: Transforms datasets between OpenAI chat, SageMaker SFT/DPO/RLVR/RLAIF, HuggingFace preference, Bedrock Nova, VERL, and custom JSONL formats.
- Validated Code Generation: Produces a
transform_dataset(df) pandas function plus an execution script, tested against sample records before running on the full dataset.
- Scalable Execution: Runs transformations locally for datasets under 50 MB or submits them as SageMaker Processing Jobs for larger datasets, with S3 or local input/output.
- Use Case: You have a DPO preference dataset and need it in VERL format for RLVR training on SageMaker — the Skill inspects your data, generates the conversion function, validates it on samples, and runs the full transformation to S3.
Quick Start
Ask the agent to transform your dataset into a target ML format, for example: convert my local DPO dataset at data/prefs.jsonl into VERL format and upload the result to S3.
