ROCm

Graphics cards were originally built for rendering images in computer games. They can perform very many simple calculations simultaneously, and that is exactly what is needed to train and run AI programs. For a program to be able to address a graphics card at all, it needs an intermediate layer of software: drivers, program libraries with ready-made computational building blocks, and tools for translating program code. ROCm is precisely this intermediate layer for the graphics cards and computing chips of the company AMD. The name originally stood for “Radeon Open Compute,” with Radeon being AMD’s product line. A large part of the code is openly viewable, which AMD likes to emphasize as an advantage over the competition.

AMD’s tool against Nvidia's lead

The market for AI computing chips has been dominated by Nvidia for years. This is only partly due to the hardware itself. What is decisive is CUDA, Nvidia’s own software layer, which has existed since 2007. Almost every AI research project of the last fifteen years was developed on it. Anyone who wants to buy a different graphics card today therefore not only needs to find a good chip, but also software that can work with it.

This is exactly where the economic significance of ROCm lies. AMD’s MI-series computing chips are quite competitive in terms of raw specifications, for example in memory per card. But if the software crashes, runs slower, or doesn’t know certain functions, good hardware is of no help. Analysts therefore regularly assess AMD’s chances in the AI business based on how mature ROCm currently is.

For data centers, this is about a great deal of money. Large operators do not want to be dependent on a single supplier, because that supplier then dictates prices. A second functioning platform reduces this dependency. That is why companies like Microsoft, Meta, and OpenAI have publicly announced that they will deploy AMD chips.

From the library to the driver

ROCm consists of several layers stacked on top of each other. At the bottom sits the driver, i.e. the program that communicates directly with the chip. Above that lies an interface with which developers can send their own computing tasks to the graphics card. It is called HIP and is intentionally designed to be very similar to CUDA. Existing CUDA code can therefore often be translated with manageable effort, instead of being rewritten completely from scratch.

Above this interface lie ready-made libraries for tasks that constantly occur in AI. The most important type of calculation is matrix multiplication, i.e. the computation of large tables of numbers. Nobody needs to program such building blocks themselves; they come bundled with ROCm. At the very top sit the well-known AI frameworks such as PyTorch, which researchers actually work with. These frameworks now officially support ROCm.

A common misconception is that ROCm is simply a graphics driver. A driver is only the lowest part of it. Conversely, ROCm is also not AI software in the narrower sense: it doesn’t learn anything and doesn’t answer any questions. It is a mediating layer, comparable to a translator between a program and the hardware. A well-known point of criticism for a long time was that ROCm officially supported only a few card models, while CUDA runs on practically every Nvidia card.

ROCm in the news and on your own computer

In business news, ROCm usually comes up when AMD unveils new computing chips or publishes quarterly figures. Typical statements are that a new ROCm version runs certain models faster or that a framework is now better supported. Such reports move the stock price because they say something about whether AMD can lure customers away from Nvidia.

In practice, one encounters ROCm anywhere AI models run on AMD hardware. This applies to large supercomputers like Frontier in the USA, but also to ordinary users. Anyone who wants to run a language-model chatbot on their own PC with an AMD graphics card installs ROCm for this purpose. On Windows, this has only been possible without workarounds for a relatively short time; on Linux it has run stably for longer.

It is also important to know that ROCm is not the only alternative to CUDA. Apple uses its own solution called Metal for its chips, and there are cross-vendor approaches such as OpenCL or Triton. So the competition does not only take place at the chip level, but at least as much in the surrounding software.