Packing Binary Is Fun
7 hours ago
- The author was motivated by criticism of JSON storage and decided to create a custom binary format.
- They built a binary packing system using variable-length integers (LEB128) and headers with type, length, and field number.
- A schema language was designed to define data structures, avoiding the need to store type information per value.
- The implementation included a lexer, parser, AST, semantic analyzer, dynamic packer, and multi-language code generator.
- The binary format achieved up to 80% reduction in file size compared to JSON, as demonstrated with a game character example.
- The project resulted in a CLI tool that can generate C/Python code and pack JSON into binary using the schema.