Mach-100 chip

The Mach-100 chip is a computer chip built specifically for artificial intelligence. It was developed by the California start-up d-Matrix, which aims to challenge market leader Nvidia with it. The chip is not intended to train an AI system, meaning to teach it something. It is meant to use a finished system: a question comes in, an answer goes out. That is exactly what happens millions of times a day when people talk to a chat program. The Mach-100 is supposed to consume less power for this than the standard graphics cards that currently do this work.

The power hunger of data centers

When an AI model responds, this is called inference. Inference is continuous operation: it occurs anew with every single request. Training, by contrast, happens once, but is very resource-intensive. Experts estimate that, in the long run, operators spend significantly more money on inference than on training. Every watt-hour saved per response therefore counts immediately in hard cash.

On top of that, there is a concrete problem: data centers are hitting the limits of power grids. In some regions of the US and Ireland, new facilities are no longer being approved because there simply is no free grid capacity. A chip that accomplishes the same work with less energy therefore doesn’t just solve a cost problem. It also helps determine whether a data center can be built at all.

Economically, the Mach-100 is also a symbol. Nvidia controls a very large share of the market for AI chips and can charge correspondingly high prices. Major customers such as Microsoft or Meta are therefore actively looking for alternatives. Start-ups like d-Matrix, Groq, or Cerebras benefit from this search, even though their market shares remain small so far.

Computing directly in memory

In a normal computer, the processing unit and memory are separate. The numbers sit in memory, computation happens elsewhere, so the data constantly has to travel back and forth between the two. With AI models, this involves enormous amounts of data. This transport costs more energy than the computation itself. Experts call this bottleneck the memory wall.

The Mach-100 pursues a different approach, so-called digital in-memory computing. The computing circuits sit directly next to or inside the memory cells. The numbers therefore barely need to be moved at all. A comparison: instead of carrying ingredients from the basement to the kitchen, you cook right in the pantry. The journey disappears, and with it most of the effort.

A second trick is quantization. Here, the chip stores the model’s numbers more coarsely, for instance with four decimal places of precision instead of sixteen. This saves space and computing time, but costs a bit of quality. A common misconception is that such chips are simply faster graphics cards. They are more like specialized tools: very good at one task, useless for many others.

Who buys chips like this

As a private individual, you will not buy a Mach-100. Chips like this sit on plug-in cards in the server racks of data centers. Anyone running a chatbot or an AI-powered search function rents computing time there. You will, at most, notice something indirectly when an answer arrives faster or a service becomes cheaper.

In business news, the name usually appears in two contexts. First, in funding rounds: d-Matrix has raised several hundred million dollars from investors, including Microsoft. Second, in reports about whether Nvidia’s dominance is crumbling. There, the Mach-100 is one example among several, not a revolution.

Important for putting this in perspective: announced performance figures almost always come from the manufacturer itself. Independent comparative tests, so-called benchmarks, often only follow years later. And even a technically superior chip only succeeds if the matching software exists for it. This is exactly where many Nvidia challengers have already failed.