>"The Photonic Fabric Module (PFM) is a multi-chip module (MCM) that incorporates an advanced TSMC 5nm ASIC (comprised of an 8 Tbps switch, 2x HBM3e controllers, and 4x DDR5 controllers), a photonic interposer (PIC), 2 HBM3e stacks, and a Fiber Array Unit (FAU) in a 2.5D package with 7.2 Tbps of optical connectivity. The HBM acts as a cache to the DDR5 memory; thus, from a processor’s perspective, it sees the bandwidth of the HBM but the capacity of the DDR5."
Simpler translation:
For any device that currently uses glued-to-it, can't be removed, can't be upgraded attached HBM or GDDR, i.e., AI Accelerator and Graphics Cards, why not:
1) Replace the multi-pin/multi-wire, heat-generating electronic connection between that glued-on, non-removable memory with a high-speed optical connection (7.2 Tbps in the above case) to memory that is decoupled from that AI Accelerator or Graphics Card, which can be separately powered, separately heat managed, etc.
2) While you're at it, make the HBM which is high speed act as an intelligent cache for additional DDR/GDDR, which although very fast, is still slower speed than the HBM. (Companies creating similar such RAM expansion devices in the future could also opt to integrate still slower speed Flash (but with much larger capacity) memory into that hierarchy)
3) Net effect: Imagine taking the HBM memory off of every AI accelerator that currently uses it, and placing it in the general vicinity around it, connecting it via optical fibers. The net result should be much better thermal management, easy upgradability (i.e., new memory is invented, but you want to keep the old compute... what to do? Just detach the old memory fiber optics and plug in the fiber optics for the new memory!). This should also result in the ability to add huge swaths of memory (memory amounts the likes of which have never been seen before) to any given AI Accelerator card or engine...
In short, a set of really good ideas here, from Celestial AI!
>"The Photonic Fabric Module (PFM) is a multi-chip module (MCM) that incorporates an advanced TSMC 5nm ASIC (comprised of an 8 Tbps switch, 2x HBM3e controllers, and 4x DDR5 controllers), a photonic interposer (PIC), 2 HBM3e stacks, and a Fiber Array Unit (FAU) in a 2.5D package with 7.2 Tbps of optical connectivity. The HBM acts as a cache to the DDR5 memory; thus, from a processor’s perspective, it sees the bandwidth of the HBM but the capacity of the DDR5."
Simpler translation:
For any device that currently uses glued-to-it, can't be removed, can't be upgraded attached HBM or GDDR, i.e., AI Accelerator and Graphics Cards, why not:
1) Replace the multi-pin/multi-wire, heat-generating electronic connection between that glued-on, non-removable memory with a high-speed optical connection (7.2 Tbps in the above case) to memory that is decoupled from that AI Accelerator or Graphics Card, which can be separately powered, separately heat managed, etc.
2) While you're at it, make the HBM which is high speed act as an intelligent cache for additional DDR/GDDR, which although very fast, is still slower speed than the HBM. (Companies creating similar such RAM expansion devices in the future could also opt to integrate still slower speed Flash (but with much larger capacity) memory into that hierarchy)
3) Net effect: Imagine taking the HBM memory off of every AI accelerator that currently uses it, and placing it in the general vicinity around it, connecting it via optical fibers. The net result should be much better thermal management, easy upgradability (i.e., new memory is invented, but you want to keep the old compute... what to do? Just detach the old memory fiber optics and plug in the fiber optics for the new memory!). This should also result in the ability to add huge swaths of memory (memory amounts the likes of which have never been seen before) to any given AI Accelerator card or engine...
In short, a set of really good ideas here, from Celestial AI!