Sick: Indexed deduplicated binary storage for JSON-like data structures
6 months ago
- #Binary Encoding
- #Data Processing
- #JSON
- SICK is an approach for handling JSON-like structures efficiently.
- Allows storing JSON data in indexed binary form for efficient access.
- Supports streaming parsers and updates for JSON-like data.
- Addresses inefficiencies in traditional JSON parsing like memory spikes.
- Uses a flattened, deduplicated table representation for JSON structures.
- Enables features like circular references and custom data types.
- Currently implemented in C# and Scala with potential for streaming support.
- Defines specific type markers for various data types in binary encoding.
- Has limitations like maximum object size and key order not preserved.