Schematischer Aufbau einer GPU: Die Chip-Fläche ist in mehrere Streaming Multiprocessors (SMs) unterteilt, jeder SM enthält eine Gruppe von Rechenkernen (CUDA Cores), Tensor Cores und einen gemeinsamen Zwischenspeicher (Shared Memory). Pfeile zeigen, wie Rechenaufgaben (Warps) vom SM-Scheduler auf die Rechenkerne verteilt werden.

Streaming Multiprocessor

A Streaming Multiprocessor is the fundamental compute block of an Nvidia graphics processing unit (GPU): it bundles many small compute cores and executes thousands of tasks simultaneously. Streaming Multiprocessors are critical for AI training and graphics rendering because a GPU's performance depends directly on how many and how powerful the SMs it contains are.

A modern graphics card (GPU) does not consist of a single compute core, but of thousands. However, these cores are not scattered individually — they are grouped together. Each of these groups is called a Streaming Multiprocessor, or SM for short. Depending on the model, an SM contains between 64 and 128 such small compute cores, along with a shared cache and the necessary control logic. The entire GPU then consists of dozens to hundreds of these SMs. A current-generation Nvidia chip — such as the H100, which is used in AI data centers — has over 130 SMs, meaning more than 16,000 individual cores in total.

The foundation of GPU performance

Why not simply build a very fast single core? Because many tasks in graphics and AI can be broken down into thousands of independent subtasks. Rendering an image, for example, means performing a very similar calculation for each of millions of pixels. Handling all these calculations simultaneously is faster than processing them one after another — no matter how fast the single core would be.

The SM is the building block that implements this parallelism. It literally executes many compute operations in the same clock cycle. The more SMs a GPU has, the more operations run simultaneously — and the higher the computing performance in the unit TFLOPS (trillions of floating-point operations per second). This figure appears again and again in GPU datasheets and is directly tied to the number and clock frequency of the SMs.

What happens inside an SM

An SM divides the tasks assigned to it into small groups of 32 threads. A thread is the smallest unit of execution — you can think of it as a single running calculation. Nvidia calls this group of 32 threads a warp. All 32 threads of a warp execute exactly the same instruction in every clock cycle, but on different data. This principle is called SIMD (Single Instruction, Multiple Data): one instruction, many data sets.

The SM often manages several warps simultaneously. If a warp has to wait briefly — because it needs data from the card’s slow main memory — the SM immediately switches to another warp that is already able to compute. This keeps the SM’s compute capacity utilized almost all the time, instead of waiting for data. This rapid switching is a key reason why GPUs are so efficient at parallel tasks.

In addition to the cores for ordinary decimal numbers, newer SMs also contain specialized units: so-called Tensor Cores. These are specifically designed for matrix multiplication — precisely the computational operation that occurs most frequently when training and running AI models. A Tensor Core completes the same task in a fraction of the time a normal compute core would need.

Streaming Multiprocessors in practice

When tech media report on new AI chips, the SM count is one of the first figures mentioned. The Nvidia H100 chip, which companies like Google, Microsoft, or Meta purchase in large quantities, has 132 SMs. Its successor, the B200, brings even more SMs with an improved architecture. Comparing these figures gives a quick sense of how much more computing power a new generation delivers.

Everyone who has ever played games has also had indirect contact with SMs. Every frame of a 3D game is created by dozens of SMs simultaneously calculating light, shadows, and surfaces. Without this parallel structure, smooth gaming at high resolution would not be possible.

A common misconception: many people confuse the number of SMs with the number of compute cores. The cores are the smallest unit; SMs are the organizing layer above them. When a GPU is advertised as a “16,384-core GPU,” this refers to the individual cores within all SMs added together — not the SMs themselves. Both figures say something about performance, but they describe different levels of the chip architecture.

Related Products

Latest News

Subscribe free. Unsubscribe the second it sucks.

High-signal news across AI, business, UX, and tech. Every morning.