Chiplet Architecture

In the past, a processor was always a single, continuous block of silicon. Today there is an alternative: several smaller building blocks are manufactured separately and then connected on a carrier to form an overall chip. These smaller building blocks are called chiplets, and the underlying principle is called chiplet architecture. To the software, the result looks like a single processor – but internally it consists of several independent parts. This may initially sound cumbersome, but it solves a fundamental problem in semiconductor manufacturing.

The size problem in chip manufacturing

The larger a chip is, the more likely it is that a tiny defect will end up somewhere inside it during manufacturing. A defective chip is scrap. With a very large chip, the chance of such a defect rises sharply, and with it the proportion of unusable products. This drives costs up massively.

Small chiplets have a much smaller area, so far less scrap results. Even if a chiplet is defective, only that one part is discarded – not the entire, expensively manufactured block. In addition, different chiplets can be produced using different manufacturing processes. For each building block, one can choose exactly the process that makes the most sense for it, instead of forcing everything into a single process.

How chiplets grow together

The individual chiplets are placed next to each other on a substrate – a kind of carrier board – and electrically connected. This connection is the critical point. Data must flow back and forth between the chiplets very quickly, without losing much energy in the process. Special connection techniques exist for this purpose, for example densely packed traces directly on the substrate or so-called interposers, i.e. intermediate layers made of silicon that couple the chiplets especially tightly together.

A concrete example: AMD's EPYC processors for data centers consist of up to eight compute-unit chiplets sitting on a shared carrier. Each compute unit is manufactured separately, using the most advanced process available. The input/output chip, which establishes the connection to the outside, on the other hand, uses an older, cheaper process. This combination saves costs without sacrificing computing performance. A monolithic chip – that is, a single large block – of this size would simply be too expensive to manufacture.

One important challenge remains: the connection between two chiplets is slower than the connection within a single chip. Engineers must therefore carefully plan which parts of the processor communicate with each other frequently – and house these, as far as possible, in the same chiplet.

Chiplets in products and headlines

Chiplet architectures are no longer a niche topic. AMD has consistently used the principle in its Ryzen and EPYC processors since 2019, and has at times overtaken Intel in computing performance as a result. Intel’s current-generation Core Ultra processors also use chiplets. Apple's M series likewise combines several building blocks internally, even though Apple itself does not always use this term.

In the field of artificial intelligence, the topic is playing an increasingly important role. Graphics processors for AI training, for instance from Nvidia, are getting larger and larger – and are thus running into the same manufacturing limits that chiplets are meant to solve. Nvidia’s H100 chip is still monolithic, but the industry is openly discussing whether future AI chips will necessarily have to switch to chiplet designs in order to remain scalable at all.

The topic is also politically relevant. The US and the EU are investing billions in their own chip factories – and chiplet architectures are a strategic advantage here, because parts of a chip can be manufactured in different countries and then assembled together. Anyone who understands the principle also understands why semiconductor policy is so complex today.