Advanced Packaging

A computer chip is a paper-thin piece of silicon, barely bigger than a fingernail. To be usable at all, it has to be placed into a housing with connectors and a cooling surface. This step is called packaging. In the past, this was purely manual work at the end of production, technically not very exciting. Advanced Packaging refers to the modern variants of this: several pieces of silicon are placed side by side or on top of each other in a shared housing and connected by thousands of extremely fine wires. The result looks from the outside like a single large chip, but internally consists of several parts.

Why packaging became the bottleneck for AI chips

For decades, chips got faster because their components kept shrinking. That path is now stalling: the structures are only a few nanometers in size, a nanometer being one millionth of a millimeter. Every further step costs billions and yields less benefit than before. So the industry is now extracting performance elsewhere: not from the individual chip, but from the clever assembly of many chips.

With AI computations, there’s a second problem. The compute units are usually faster than the data can be delivered from memory. So they wait instead of computing. Advanced Packaging places the memory directly next to or on top of the compute unit. The distances shrink from centimeters to millimeters, allowing significantly more data to flow per second.

This has economically shifted the chip market. Capacity for these processes is scarce, especially at the Taiwanese contract manufacturer TSMC. Anyone who cannot secure a slot in its packaging lines cannot ship their AI accelerators, even if the chips themselves are finished. That’s why packaging capacity now regularly comes up in quarterly reports and analyst questions.

Chiplets, stacks, and the silicon interposer

The basic idea is called chiplet design. Instead of manufacturing one huge chip in a single piece, several smaller partial chips are built. This is cheaper, because manufacturing always produces defects. On a large chip, a single defect renders the entire piece worthless; with small parts, only one part is lost. The good chiplets are then assembled together.

There are two basic forms of connection. In the flat variant, the chiplets sit side by side on a thin silicon interposer. It contains the fine wiring traces and acts like an extremely densely wired circuit board. This design is called 2.5D, because although it is multi-layered, the compute chips all lie in one plane. TSMC’s well-known process for this is called CoWoS.

In true 3D packaging, chips lie directly on top of each other. They are connected by vertical channels that run through the silicon. This is also how the stacked memory in modern AI cards is created, with eight or more memory layers stacked on top of each other. The price for this is heat: chips stacked on top of one another heat each other up, and cooling struggles to reach the lower layers. As a result, the defect rate is high, and every flaw destroys an already expensive component.

From graphics cards to smartphone processors

Advanced Packaging is most visible in AI data centers. The accelerator cards from Nvidia or AMD consist of compute units and several memory stacks on a shared interposer. Such a module is roughly the size of a beer coaster and often costs more than a small car. Without this design, today’s large language models would not exist in their current form.

The technology is also found in everyday devices. In smartphones, memory and processor are often placed on top of each other to save space. Apple's laptop chips connect several components within one housing. If you read in a review that a processor consists of several “tiles” or “dies,” you are reading about chiplets.

A common misconception: Advanced Packaging does not automatically make a chip more advanced in terms of manufacturing size. A chiplet made using a 5-nanometer process remains 5 nanometers, no matter how cleverly it is packaged. Packaging determines data throughput, form factor, cost, and yield. But these exact factors currently limit how many AI chips can be built at all.