TernFS – An exabyte scale, multi-region distributed filesystem
7 hours ago
- #distributed-filesystem
- #big-data
- #storage
- XTX developed TernFS, an exabyte-scale, multi-region distributed filesystem, to meet its growing storage needs.
- TernFS is designed to scale up to tens of exabytes, trillions of files, and millions of concurrent clients.
- Key features include redundancy against drive failures, no single point of failure in metadata services, and support for file snapshots.
- TernFS spans across multiple regions, is hardware agnostic, and utilizes different types of storage cost-effectively.
- Files in TernFS are immutable, and the system is optimized for files with a median size of 2MB.
- The filesystem's core API is implemented by four services: Metadata shards, the cross-directory coordinator (CDC), Block services, and the Registry.
- TernFS uses Reed-Solomon coding for redundancy and CRC32-C checksums to protect against bitrot.
- The system includes a scrubbing process to continuously verify data integrity and a garbage collector to manage expired snapshots.
- TernFS is open-sourced and available on GitHub, with a focus on simplicity, performance, and reliability.