So reflecting on that, I think the core assumption that didn't pan out is that the memory / CPU ratio will grow because we'll need more memory, hence requiring CPUs to address more than 16 EiB of data (16 EiB = 16384 PiB = 16777216 TiB of data, what 64 bits can address).
But in practice, we've produced a lot more compute. The memory / CPU ratio has increased, but most growth has been from more CPUs (and generally not shared-memory ones, but ones with their own completely separate memory space).
IPv6 is 128 bit, so we do have 128 bit ways of addressing computers, but I'd say we're still a long way from a CPU needing to address that much memory as a common case. The speed of light limits how far away memory can be from the CPU for good performance, so short of some drastically new memory technology, it seems unlikely we'll need it soon for any ordinary type of computing device.
Also arguably we already have 256 or 512 bit CPUs, what matters most for memory throughput is not the register width but essentialy the L1 cache line width (eg what in old CPUs was the databus width).
As for address width, we're not even close to get full 64 bit pointers from CPUs anyway, more like 48 or 52 bits.
Wide pointers (eg 128 bit general registers) would make sense for carrying capabilities for memory safety though I guess.
Personally I'm disappointed 128-bit floating point (quadruple precision) never properly made it into CPU's (sure it appeared in some niche ones here and there, but not in what we actually use today). After all, in the 1980's they had 80-bit ones, it's not even that far off, and they had millions times less transistors then.
While probably niche and applications that need higher precision using their own custom types anyway, they'd allow cool stuff like easy to program fractals with much higher detail than now. But also, less precision loss in many applications.
That's the thing, bigger datatypes means less effective memory throughput. From that perspective 16-bit or even 8-bit floats are often more useful than 80 or 128 bit floats. Same problem with 64 bit pointers and why it's often better to store narrower indices instead of full pointers, data can be packed more tightly and accessed more efficiently.
The point about large word lengths is you get higher data throughput, and faster processing, because everything is going through fat pipes into a big parallel machine with multiple levels of cache and vectorisation.
The issue is the utility of floats at different precisions. 128-bit floats have some benefits for high-end scientific applications, but the extra cost and complexity over 64-bit hardware would only make sense for specialised scientific supercomputing. So far it just hasn't been worth it.
So reflecting on that, I think the core assumption that didn't pan out is that the memory / CPU ratio will grow because we'll need more memory, hence requiring CPUs to address more than 16 EiB of data (16 EiB = 16384 PiB = 16777216 TiB of data, what 64 bits can address).
But in practice, we've produced a lot more compute. The memory / CPU ratio has increased, but most growth has been from more CPUs (and generally not shared-memory ones, but ones with their own completely separate memory space).
IPv6 is 128 bit, so we do have 128 bit ways of addressing computers, but I'd say we're still a long way from a CPU needing to address that much memory as a common case. The speed of light limits how far away memory can be from the CPU for good performance, so short of some drastically new memory technology, it seems unlikely we'll need it soon for any ordinary type of computing device.
Because of LLMs we are back to memory being king (KV cache).
But there was another thing - clusters of thousands/millions of machine instead of one big iron with all the memory.
the cases where its useful, it's in vectorized instruction sets, etc.
You're conflating 128-bit registers with 128-bit memory addressing. This article is about the latter.
Also arguably we already have 256 or 512 bit CPUs, what matters most for memory throughput is not the register width but essentialy the L1 cache line width (eg what in old CPUs was the databus width).
As for address width, we're not even close to get full 64 bit pointers from CPUs anyway, more like 48 or 52 bits.
Wide pointers (eg 128 bit general registers) would make sense for carrying capabilities for memory safety though I guess.
Personally I'm disappointed 128-bit floating point (quadruple precision) never properly made it into CPU's (sure it appeared in some niche ones here and there, but not in what we actually use today). After all, in the 1980's they had 80-bit ones, it's not even that far off, and they had millions times less transistors then.
While probably niche and applications that need higher precision using their own custom types anyway, they'd allow cool stuff like easy to program fractals with much higher detail than now. But also, less precision loss in many applications.
That's the thing, bigger datatypes means less effective memory throughput. From that perspective 16-bit or even 8-bit floats are often more useful than 80 or 128 bit floats. Same problem with 64 bit pointers and why it's often better to store narrower indices instead of full pointers, data can be packed more tightly and accessed more efficiently.
The point about large word lengths is you get higher data throughput, and faster processing, because everything is going through fat pipes into a big parallel machine with multiple levels of cache and vectorisation.
The issue is the utility of floats at different precisions. 128-bit floats have some benefits for high-end scientific applications, but the extra cost and complexity over 64-bit hardware would only make sense for specialised scientific supercomputing. So far it just hasn't been worth it.