tokf-filter

TOML filter files define transformation rules for the tokf CLI tool to compress command-line output. They use step types like skip, replace, and section to extract only the relevant output, significantly reducing token consumption for AI agents.

To create a TOML filter, define your command and use step types like [[section]] and [[replace]] to isolate specific output segments. This allows you to extract only the final summary from commands like cargo test or git status for LLM context.

Filter steps like skip, replace, and section define specific transformation actions for command-line output. Skip omits lines, replace modifies text patterns, and section extracts specific blocks, allowing precise control over what enters the LLM context window.

Yes, TOML filters support a [json] step type to parse and transform command-line JSON output. This allows you to extract specific fields or restructure the data before passing it as context to an LLM, reducing token waste.

The best way to explain complex filter logic is to clarify the purpose of each TOML step type and how they interact to transform output. Understanding this structure helps ensure your filter accurately captures the necessary command-line data for LLM context.

You should use output compression filters when a command generates verbose output that exceeds LLM token limits or adds unnecessary cost. Filters isolate essential information like test summaries, ensuring efficient and focused context delivery to the AI agent.