skills.rest
yekinya
yekinyaCOMMUNITY
Explore all skills by this author →
GitHub

moai-formats-data

Encode, validate, and optimize JSON, YAML, and TOON data formats for LLM transmission.

2|Updated May 27, 2026
Search Tags:#json#yaml#data validation#token optimization#serialization#toon#orjson
One-click install
npx skills add https://github.com/yekinya/moai-novel --skill moai-formats-data-yekinya
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: moai-formats-data
Source: https://github.com/yekinya/moai-novel/tree/main/moai-novel/.claude/skills/moai-formats-data
Command: npx skills add https://github.com/yekinya/moai-novel --skill moai-formats-data-yekinya

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires orjson, PyYAML, ijson, jsonschema, python-dateutil, and includes references (resource) components.

What problem does it solve? Sending structured data to LLMs wastes tokens on verbose JSON syntax, and unvalidated or inconsistently serialized data causes silent runtime errors. This Skill provides TOON encoding for 40-60% token reduction, high-performance JSON/YAML processing, and schema-based data validation. ## Core Features & Use Cases - TOON Encoding: Compress structured data with type markers (#, !, @, ~) for lossless round-trip encoding that reduces LLM token usage by 40-60% versus JSON. - High-Performance Serialization: Use orjson for 2-5x faster JSON processing, ijson for streaming large files without memory exhaustion, and PyYAML C loaders for configuration management. - Data Validation & Schema Evolution: Validate data against typed schemas with custom rules, cross-field checks, and versioned migration support. - Use Case: When building an API that feeds responses to an LLM, encode payloads with TOONEncoder to fit token budgets, validate inputs with DataValidator schemas, and stream multi-hundred-megabyte JSON files with StreamProcessor. ## Quick Start Ask the AI to encode a sample user dictionary with TOONEncoder and compare the token count against standard JSON output.

Frequently Asked Questions about moai-formats-data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce token usage when sending JSON data to an LLM?▼

Encode the data with TOON (Token-Optimized Object Notation), which uses type markers like # for numbers and ! for booleans to remove JSON syntax overhead. This achieves 40-60% token reduction with lossless round-trip decoding.

What is the fastest Python library for JSON serialization?▼

orjson serializes and deserializes JSON 2-5x faster than the standard json module and produces bytes output directly. It also supports numpy, dataclasses, and UUID serialization natively.

How do I process large JSON files without running out of memory?▼

Use ijson to stream-parse large JSON files item by item instead of loading the entire file into memory. Combine it with batch processing to handle files of several hundred megabytes with constant memory usage.

When should I use TOON instead of JSON or YAML?▼

Use TOON only for LLM communication where token budgets matter; it is not human-readable and unsuitable for long-term storage. Use JSON for machine data exchange and YAML for human-edited configuration files.

How do I validate data against a schema in Python?▼

Create a schema defining field types, required flags, length and value ranges, and regex patterns, then call validate with the data and schema. The result reports validity, per-field errors, and sanitized data with type coercion applied.

Why does TOON decoding fail with an invalid type marker error?▼

Decoding fails when the input contains unsupported data types, the encoder and decoder use different configurations, or the string was corrupted in transit. Wrap decode calls in error handling with a JSON fallback.

Related Skills

Explore More Software Engineering Skills→
TheBushidoCollectiveTheBushidoCollective
187

django-rest-framework

Create Django REST Framework serializers, viewsets, and authentication for RESTful APIs.

Official
Intermediate
cenjiecenjie

api-response-optimization

Optimize REST and GraphQL API responses with HTTP caching, compression, and payload minimization.

Community
Intermediate
DazlarusDazlarus

time-handling

Reviews datetime code for timezone awareness, UTC storage, and DST-safe calculations.

Community
Intermediate
AGIBuildAGIBuild

serialization

Implements JSON, Protobuf, and MessagePack serialization patterns for AOT-compatible .NET applications.

Community
Intermediate