GPU

A processor — the “brain” of a computer — handles tasks one after another: step by step, very quickly. A GPU, short for Graphics Processing Unit, works differently. It doesn’t have one fast core, but thousands of smaller ones that compute simultaneously. Originally, this was intended for video games: a screen displays millions of pixels, and all of them need to be calculated at the same time. Today, this property is used primarily for AI. Training a large language model — meaning learning from enormous amounts of text — consists of billions of computational operations that can be split up well. The GPU handles them in parallel, which means weeks instead of years.

Why the GPU made the AI breakthrough possible in the first place

About twelve years ago, researchers tried to train neural networks — learning systems roughly modeled on the human brain — on conventional processors. This took too long, and the models remained small. In 2012, a team from the University of Toronto showed that the same model, trained on GPUs, was finished in days instead of months. This moment is considered the starting signal of the modern AI era.

Since then, the demand for GPUs has multiplied many times over. Large models like GPT-4 were trained on clusters of tens of thousands of GPUs. Nvidia, the market leader in AI-capable GPUs, increased its stock market value in just a few years from around 300 billion to over 3 trillion US dollars — a direct reflection of how central these chips have become.

Why parallel computing suits AI so well

At the heart of AI training is a computational operation called matrix multiplication. This involves multiplying large tables of numbers together. These tables can be broken down into many small subtasks that are independent of one another. A GPU with thousands of cores handles these subtasks simultaneously — a conventional processor would work through them one at a time.

Nvidia’s GPUs for the AI market — such as the H100 or B200 series — are also equipped with very fast memory, because data needs to be delivered quickly enough so the cores don’t have to wait for it. A single such GPU costs between 25,000 and 40,000 US dollars. Anyone operating many of them refers to it as a data center or GPU cluster. The major AI labs such as OpenAI, Google DeepMind, or Anthropic operate such clusters with tens of thousands of units.

There is an important difference between training and operation: during training, the model learns and needs very many GPUs simultaneously. During operation — that is, when someone asks a question and receives an answer — fewer chips are often sufficient. This second step is called inference and determines how expensive each individual response is for the provider.

GPUs in the news and everyday life

In business news, GPUs often appear in connection with export controls. Since 2022, the US government has banned the export of certain high-performance GPUs to China — out of concern that they could be used for military AI systems. As a result, Nvidia had to develop scaled-down versions of its chips that comply with these rules.

In everyday life, a GPU is found in almost every laptop and smartphone — there, it’s usually responsible for video playback or games, not AI training. When services like ChatGPT, Midjourney, or Google Gemini are used, the actual computation runs on GPUs in data centers far away. One’s own device only sends the request and receives the response.

A common misconception: many people think that a better GPU in their own computer would make AI models smarter. In fact, one’s own hardware only affects how quickly the model responds when it runs locally — meaning directly on one’s own machine, without an internet connection. The model’s capabilities are determined by the training that took place long beforehand on massive clusters.