Hot Chips 3D DRAM session: D-Matrix says its 3D DRAM moves data 6-to-7x more efficiently than HBM4 on measured silicon "On comparing with HBM4, like we heard in the morning, we can get about 2.4 picojoules per bit energy efficiency on HBM4 of moving data into the base die." "There's obviously additional energy burned to move the data sideways into the GPU, which is not counted here." "But using this face-to-face stacking with 3D DRAM, we are able to reduce that energy to 0.37 picojoules per bit. This is a measured number." "And so this gives us, you know, 6 to 7X energy efficiency." "So for the same amount of power you can drive the bandwidth up, and so we were able to drive the bandwidth up here to 100 terabytes per second." "And in this construction it's a single-high stack, and like I showed, even with 32 gigabytes of memory capacity we are able to solve SOTA models in a scale-up network." "So no bits are wasted."

Nvidia and AI chip startup d-Matrix are combining their hardware in a new system to power AI models, the latest example of Nvidia partnering with rivals (@_pheebini / The Information) (Visit Techmeme dot com for the link and full context!)