Co-Design

When two parts of a system need to work together later, you can build them one after the other or at the same time. The usual way is sequential: first the machine is built, then someone writes the program for it. Co-design does it differently. Both sides are designed together from the very beginning, and any decision on one side is allowed to change the other. In computer technology, this usually concerns the chip and the software that computes on it. The result is a system whose parts fit together precisely – but also one that is harder to take apart.

Why chips and programs hold each other back

A chip is a piece of silicon with hard-wired computing units. What it cannot do, it cannot learn to do later either. If a chip is fully designed before it is clear which programs will run on it, losses arise almost inevitably. The chip masters things nobody needs. And it lacks capabilities that the program would urgently need.

With AI systems, this loss is especially costly. Large language models – that is, programs that continue texts and have learned to do so from huge amounts of text – need thousands of chips for weeks to train. Power consumption reaches amounts measured in millions. If such a system is only seventy percent utilized instead of ninety percent, the gap costs real money. This is exactly where co-design comes in.

The second reason is competitive speed. Whoever designs the model and the computer together can try out ideas that would be unaffordable on standard hardware. Companies like Google, Apple, or Tesla therefore build their own chips for their own software. This is expensive, but it provides a lead that competitors using standard parts find hard to catch up with.

The cycle between design and measurement

Co-design runs in rounds. A team describes what the software needs to compute – for example, very many multiplications of number tables in sequence. The hardware team then builds a computing unit that executes exactly this operation especially fast. The software then runs on a simulation of the new chip, long before it is actually manufactured. The measurements produce new requests for changes, and the next round begins.

Often the software changes more than the hardware in this process. One example is numerical precision. Classic computers compute with very precise numbers, but AI models cope surprisingly well with coarser numbers. Anyone who knows this builds computing units that only handle these coarser numbers – but fit twice as many of them onto the same chip area. The software is thus made deliberately less precise so that the hardware can be smaller.

A common misconception is that co-design simply means good coordination between two departments. The difference runs deeper. In co-design, neither side is finished until the other is too. This costs time and binds the parts together: the chip is of little use without this software, and the software runs slowly elsewhere.

From phone cameras to AI data centers

The best-known example sits in many pockets. Apple designs the chips for its iPhones itself and tunes them to its own operating system. That’s why these devices often last longer than competing models with similarly powerful standard chips. The camera is also a case of co-design: part of the image quality does not come from the lens but from a computing unit built specifically for the image-processing software.

In the news, the term mostly appears in connection with AI data centers. Google’s TPU chips were developed together with Google’s AI software. Nvidia supplies a software package called CUDA alongside its graphics chips, tailored precisely to these chips – one reason why competitors find it so hard to catch up. When a company announces it is designing its own chips, the desire for co-design is almost always behind it.

The term is now also used outside of technology. In urban planning or product development, co-design means that future users already have a say during the design phase. The basic idea is the same: planning together instead of working through steps one after another. In a tech context, however, it almost always refers to the combination of chip and software.