token-optimizer

Compress prompts and documents to reduce token counts with Python scripts.

27|9|Updated Jan 4, 2026

Search Tags:#prose #compression #docs #token #prompt #TOON

Ready to run

npx skills add https://github.com/georgekhananaev/claude-skills-vault --skill token-optimizer

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: token-optimizer
Source: https://github.com/georgekhananaev/claude-skills-vault/tree/main/.claude/skills/token-optimizer
Command: npx skills add https://github.com/georgekhananaev/claude-skills-vault --skill token-optimizer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires tiktoken, and includes scripts (resource) components.

What problem does it solve?

This Skill reduces token count in prompts, documentation, and prose to improve LLM context efficiency and readability.

Core Features & Use Cases

Prompt Compression: dramatically reduces prompt size while preserving meaning.
Doc Formatting: compresses and standardizes Markdown/doc prose for quick comprehension.
TOON Serialization: converts data objects to TOON format to minimize context size.
Prose Clarity: applies Strunk's rules to produce concise, precise text.

Quick Start

Use the token-optimizer to compress a sample prompt or document. Example: python scripts/compress_prompt.py "Your long text here" or python scripts/compress.py docs.md

Frequently Asked Questions about token-optimizer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I reduce token count in long prompts and documentation?▼

You can reduce token count in long prompts and documentation by running Python scripts that apply Strunk's prose rules, outputting compressed text alongside token statistics to improve LLM context efficiency.

What is TOON serialization for minimizing AI context size?▼

TOON serialization is a data conversion method that minimizes context size by converting data objects into the TOON format. It is used to reduce token consumption when passing JSON or YAML data to an LLM.

Does the token-optimizer work with JSON and YAML data inputs?▼

Yes, the token-optimizer works with JSON and YAML data inputs by compressing and standardizing them for AI input. It processes these formats to minimize context size and improve LLM readability.

How do I compress a Markdown README using Python scripts?▼

To compress a Markdown README, you run the compress.py script with your target file. The script applies prose compression techniques to standardize the document and output token statistics for quick comprehension.

Do I need tiktoken installed to count tokens in my prompts?▼

You need tiktoken installed if you want to use optional support for accurate token counting in your prompts. The scripts can output compressed text and statistics, with tiktoken providing the underlying token calculation.

When should I not use prose compression for my LLM inputs?▼

You should avoid prose compression when maximum detail and exact wording are legally or technically required, because applying Strunk's rules to reduce token count intentionally removes verbose phrasing to produce concise text.