Toon – Token Oriented Object Notation
6 months ago
- #data-format
- #LLM
- #efficiency
- TOON (Token-Oriented Object Notation) is a compact, human-readable format designed for structured data transfer to LLMs with reduced token usage.
- TOON typically reduces token usage by 30–60% compared to JSON by eliminating redundant punctuation and using efficient tabular arrays.
- Key features include token efficiency, LLM-friendly guardrails, minimal syntax, indentation-based structure, and tabular arrays.
- TOON supports various delimiters (comma, tab, pipe) and includes deterministic formatting rules for consistency.
- The format is particularly effective for uniform data structures but falls back to list format for mixed or complex data.
- TOON is available as an npm package (@byjohann/toon) and includes functions like `encode` for converting JSON to TOON.
- Special cases like empty containers, non-JSON types, and quoting rules are handled to ensure compatibility and efficiency.