19 hours ago
- OCaml is a pragmatic functional language that can hide C buffer management techniques behind abstract interfaces.
- The safety of a buffer management technique was compromised when OCaml became multicore, as discovered in the ocaml-wire library.
- ocaml-wire implements EverParse's model of binary formats with validators that reject malformed input before application code sees it.
- A naive validator allocates a new array of slots on each call, leading to high allocation (0.3 GB for 10 million calls).
- The usual fix hoists the scratch array to the toplevel for reuse, eliminating allocations.
- In OCaml 5 with domains, shared scratch buffers cause data races when multiple domains validate simultaneously.
- ThreadSanitizer reveals the data race, and the fix uses Domain.DLS (domain-local storage) to give each domain its own scratch buffer.
- Domain-local storage is safe for fibers because fibers only interleave at effect points, and decode performs no effects.
- The bug existed in two places in ocaml-wire: decode scratch and parametric codecs' max_len bound, both fixed in version 1.1.0.