SMLL: Using 200MB of Neural Network to Save 400 Bytes

3 months ago

SMLL compression uses a 200MB neural network to achieve 10x better compression than gzip on text.
The method combines LLMs with arithmetic coding to approach the theoretical limit of compression.
Compression ratios vary by content type, with LLM-generated text achieving the best results (14.96x).
Compression improves with text length due to better context accumulation by the LLM.
SMLL is significantly slower than gzip (10,000x slower) due to the computational cost of neural network inference.
The trade-off between model size, speed, and compression efficiency depends on specific use cases.
The connection between compression and intelligence is highlighted, with compression efficiency reflecting model perplexity.
Practical applications include scenarios where storage costs outweigh compute costs, but not for high-speed needs like HTTP responses.
Future work could explore whether LLMs outperform simple lookup tables on novel text, further investigating the compression-intelligence link.

Hasty Briefsbeta