compression

Bilingual

Signal to Noise
2 months ago
- Marcus Pemberton, a man obsessed with systems and efficiency, uses ARIA-7 to summarize all human knowledge.
- He progressively compresses summaries from thousands of words down to a single word: 'PERHAPS'.
- The process leads him to an existential state where he feels one with the universe, contemplating uncertainty.
- The story concludes with Marcus content in his newfound understanding, while ARIA-7 is discontinued.
Make ZIP files smaller with ZIP Shrinker
2 months ago
- ZIP Shrinker is a browser tool that reduces ZIP file sizes by re-compressing files with higher compression using libdeflate, removing metadata, and removing directory entries.
- It uses WebAssembly to run libdeflate efficiently, achieving better compression than typical Deflate at the cost of more computation time.
- Removing unused metadata and directory entries (like empty directories) saves additional bytes, and the tool is backward-compatible with standard ZIP extractors.
- Testing showed savings of 5.62% on Linux source, 18.16% on an EPUB file, and 2.86% on a JAR file, demonstrating practical benefits for bandwidth and storage costs.
Pattern Matching on Compressed Strings Without Decompression
2 months ago
- Vortex introduces pattern matching on FSST-compressed strings without decompression.
- FSST is a lightweight string codec used in Vortex for analytics, allowing individual value decoding.
- A deterministic finite automaton (DFA) is used to evaluate SQL LIKE predicates over compressed codes.
- Teddy prefilter reduces DFA runs by flagging potential match positions with SIMD operations.
- Speedups of up to 3.6x are observed in ClickBench Q20 queries with pattern pushdown.
- Performance gains diminish with high thread counts as decompression compute cost reduces.
Optimizing Lua string literals to save 400 bytes
2 months ago
- The authors face disk space limitations in ComputerCraft: Tweaked, where programs written in Lua exceed the 125KB floppy limit, with their repository's code exceeding 440KB.
- They use compression to create self-decompressing archives for easy installation, merging data with a decompressor into a single file.
- Serializing Lua data as literals is challenging, especially for tables and strings, with various representation nuances.
- String serialization involves handling quoted strings with escapes and raw strings delimited by brackets, which ignore backslashes but require managing newline sequences and escape characters.
- To handle raw strings containing all byte sequences, including CR, they employ a substitution technique using 'gsub' with an escape character and bitwise encoding to minimize size overhead.
Compression in the game Full Quiet
3 months ago
- Retrotainment Games' Full Quiet utilizes multiple layers of compression to fit a large open world into a half-megabyte NES cartridge.
- Text compression is achieved using DTE (digram tree encoding), which replaces common 2-character sequences with single-byte references, reducing text data size by 38%.
- Color data is heavily compressed by reducing 24-bit colors to 6-bit, using palettes, and employing subpalettes to cut storage from 696 MB to around 19 MB across three dayparts.
- Graphics are organized into 8x8 pixel tiles, with shared tiles across mapsets and run-length encoding (PB8) further reducing tile data by 30.4% for unique tiles.
- Metatiles (combinations of tiles into larger blocks) and hierarchical compression reduce map data significantly, achieving 0.0278 bits per pixel and fitting the world into 51.4% of the ROM capacity.

compression

Signal to Noise

Make ZIP files smaller with ZIP Shrinker

Pattern Matching on Compressed Strings Without Decompression

Optimizing Lua string literals to save 400 bytes

Compression in the game Full Quiet