- HBF (high-bandwidth flash) stacks NAND flash layers to deliver SSD-like capacities (up to 512 GB per module) with HBM-like bandwidth (1.6 TB/s initially, targeting 3.2 TB/s eventually).
- HBF is designed to supplement HBM for AI inference, not replace it, because NAND has limited write endurance and microsecond latency vs. nanosecond for DRAM.
- In inference, HBF excels at the read-heavy decode phase (write-once, read-many), while HBM handles write-intensive prefill, making model weights readily available without reloading.
- Larger models like multi-trillion-parameter MoE could fit on a single accelerator using HBF, reducing chip-to-chip interconnect bottlenecks.