I remember back during the Internet bubble, companies were throwing fiber into the ground as fast as they could. They told us we needed TONS of this stuff to make the backbone of the new technology called “The Internet.” Stocks went to the moon, because the math said we needed a ton of this stuff.
They went overboard, of course. But it got WAY worse when some clever human (probably a white guy or Asian guy) figured out a way to send MANY different channels over the SAME piece of fiber by using different wavelengths (colors). Practically overnight, this increased the carrying capacity of the old fiber by between 16 and 100x.
Guess what? Now we need about 50% to 90% LESS fiber than we thought. Prices crash. Bubble bursts.
Google Research may have just done the exact same thing with memory by unveiling TurboQuant this week:
“……a compression algorithm for large language models and vector search engines, that shrinks a major inference-memory bottleneck: it reduces an AI model’s memory 6x, making it 8x faster with the same number of GPUs, all the while maintaining zero loss in accuracy and “redefining AI efficiency.”
The implication is clear: if Google can achieve the same inference results with one-sixth of the hardware, then demand for memory chips will collapse in inverse proportion – the same ravenous demand that until recently sent DDR prices as much as 7x higher in just 3 months when the memory bottleneck for AI became apparent.
Just a month or two ago, Comrade Clayton was wailing and gnashing her teeth over “AI ruining gaming because the price of memory was skyrocketing.”
Chicken Little was wrong yet again. While she cried and whined, the human brain (The Ultimate Resource!) came up with a clever solution to save the day as we have done for 5,000 years.
It’s like that Silicon Valley scene….
But wait, it gets better: because if Google has already found a compression algo that achieves such phenomenal efficiency improvements, it is virtually certain that further optimization – and competing algos – will surely lead to far greater efficiency, reducing the amount of hardware needed even further.
And just like that, suddenly the memory bubble which was built on the assumption that demand for DRAM and NAND will persist well into the future, looks set to burst as software may have just solved a very sticky hardware problem.