Pattern Matching on Compressed Strings Without Decompression
20 hours ago
- #Compression
- #Data Processing
- #String Matching
- Vortex introduces pattern matching on FSST-compressed strings without decompression.
- FSST is a lightweight string codec used in Vortex for analytics, allowing individual value decoding.
- A deterministic finite automaton (DFA) is used to evaluate SQL LIKE predicates over compressed codes.
- Teddy prefilter reduces DFA runs by flagging potential match positions with SIMD operations.
- Speedups of up to 3.6x are observed in ClickBench Q20 queries with pattern pushdown.
- Performance gains diminish with high thread counts as decompression compute cost reduces.