HBM4

Every computer needs a working memory alongside its compute chip. That’s where the data currently being processed is stored. In normal PCs, this memory sits as narrow sticks a certain distance from the compute chip on the motherboard. HBM4 does it differently: several wafer-thin memory dies are stacked on top of one another and glued directly next to the compute chip, connected via thousands of tiny wires. This allows far more data to reach the processing unit every second. The abbreviation stands for High Bandwidth Memory, meaning memory with a high data width, and the 4 denotes the fourth generation of this design.

Why AI chips starve without fast memory

An AI model consists of billions of numbers, so-called parameters. To generate a single answer, the chip has to fetch nearly all of these numbers from memory once. The computing itself is fast. Fetching is the bottleneck. Experts say that modern AI accelerators are memory-bound: they wait more than they compute.

That’s why memory speed often determines performance more directly than raw computing power. A chip with twice as many compute units achieves little if the supply of data stays the same. HBM4 is meant to roughly double the data rate per stack compared to the previous generation, HBM3E, while also offering more capacity. For data centers, this means more answers per second at the same power consumption.

Economically, HBM4 is also a bottleneck. Only three manufacturers worldwide can produce this kind of memory in large volumes: SK Hynix, Samsung, and Micron. Anyone who hasn’t secured a supply volume cannot manufacture their AI chips, even if everything else is ready. That’s why reports about HBM4 contracts regularly appear in stock market news.

The stack and the holes through the silicon

Classic memory lies flat side by side, whereas HBM4 goes upward. Up to sixteen memory dies are stacked on top of each other. Fine holes are drilled through each die and filled with metal, so-called Through-Silicon Vias. They act like elevator shafts in a high-rise building, connecting all floors vertically.

The whole stack then sits together with the compute chip on a shared carrier board, the interposer. Because the distances there are only millimeters, an extremely large number of lines can be routed in parallel. HBM4 uses 2048 data lines per stack, twice as many as HBM3E. Each individual line isn’t particularly fast on its own. It’s the sheer number that creates the bandwidth, similar to a highway with very many lanes at a moderate speed.

Also new with HBM4 is the bottommost chip in the stack, the base die. It controls the memory and is now manufactured using modern logic fabrication, sometimes even at contract manufacturers like TSMC. Customers can have it customized to their own chip. This blurs the line between memory and processor. A common misconception, by the way, is that HBM is simply faster RAM for gaming PCs: the design is too expensive and too elaborate, and it only pays off for servers and accelerators.

HBM4 in products and headlines

You won’t buy HBM4 directly. You encounter it built into AI accelerators for data centers, for instance in Nvidia's upcoming Rubin generation or in competing products from AMD. Every time a chatbot answers or an image generator creates a picture, this kind of stacked memory is very likely working in the background.

In business news, the term mainly appears in two contexts. First, in quarterly results of memory manufacturers, whose profits depend heavily on HBM. Second, in reports about supply shortages, since production capacity is often sold out years in advance. Prices for ordinary PC memory are also rising as a result, because factories are shifting their capacity toward the more lucrative HBM.

Important for putting reports into context: HBM4 is an open standard set by the industry body JEDEC, finalized in 2025. Mass production is set to begin in 2026. Variants with the added E, i.e. HBM4E, are faster refinements of the same generation, not a new tier.