High-bandwidth flash progress and future

Summary

The high-bandwidth flash (HBF) market could be bigger than high-bandwidth memory (HBM) in 12 years’ time according to the leading HBF evangelist, Professor Kim Jung-ho of the School of Electrical Engineering at the Korea Advanced Institute of Science and Technology (KAIST). He revealed this and more on a talk about HBF technology at a “Materials, Components, and Equipment (So-Bu-Jang) Future Forum” held in Seoul, S Korea, on January 16. HBF progress is accelerating. As reported in the Sisa Journal, he said: “Samsung Electronics and SanDisk plan to integrate HBF into products from Nvidia, AMD, and Google by late next year or early 2028.” Professor Kim Jung-ho He conceives of High-Bandwidth Memory (HBM) acting as a very fast data cache layer for GPU accelerators with a tier of HBF feeding that, and slower but much higher capacity networked SSD storage feeding the HBF tier. Nvidia has announced an Inference Context Memory Storage Platform (ICMSP) where its Dynamo and NIXL software provides a managed memory space for AI Inferencing tokens stored in a KV cache that covers HBM and BlueField-4-connected SSDs. Intuitively the same 2-tier software could be extended to cover a 3-tier HBM to HBF to BlueField-4-connected SSDs arrangement. Networked, external storage would then connect to the BlueField-4 SSD tier and have a clear path to the GPUs with no diversion though a host X86 processor and its DRAM. The KAIST professor discussed an HBF unit having a capacity of 512 GB and a 1.638 TBps bandwidth. This capacity compares to a 2Tb (250 GB) 3D NAND die, such as one built by SK hynix using its 321-layer, triple-string 3D NAND. We would only need to stack two of these to create a 512 GB capacity HBF chip. Such a 2-layer chip would contain 642 layers of 3D NAND in total, in six component strings, and would need to be fabricated such that the upper layers did not warp the lower ones. SK hynix AIN-B diagram SK hynix is developing AIN B NAND products using HBF technology. A conceptua...