SRAM

Every computer needs a place to briefly store numbers it is currently working with. Several different kinds of memory exist for this purpose, and SRAM is the fastest of them. The abbreviation stands for Static Random Access Memory. “Random access” means the chip can directly address any stored number without having to work through others first. “Static” means the content stays stable as long as power flows and doesn’t need to be refreshed constantly. Turn the device off, and everything is gone — SRAM is no substitute for a hard drive.

SRAM almost always sits directly on the same piece of silicon as the compute units. That’s why the distances are extremely short, and a single access takes only a few billionths of a second. The price for this is space: SRAM needs a lot of chip area per stored value.

Why fast memory determines chip performance

A modern processor can carry out billions of computational steps per second. That’s only useful if the right numbers arrive in time. If the chip has to fetch every value from the more distant main memory, it spends most of its time waiting. Experts call this the memory bottleneck. SRAM is the most important countermeasure, because it keeps the most frequently used data in the immediate vicinity.

With AI models, this bottleneck is especially noticeable. A large language model consists of many billions of stored numbers, the parameters. For every answer, huge amounts of them have to flow through the compute units. The bottleneck here is usually not computing power, but how fast data can stream in. That’s exactly why chipmakers talk about memory at such length in their presentations.

SRAM, however, is expensive. It costs roughly six transistors to store a single bit — that is, a single yes/no piece of information. The slower DRAM used in main memory needs only one transistor and a tiny capacitor for the same job. That’s why a chip might have a few hundred megabytes of SRAM, but gigabytes of other memory.

Six transistors per bit

At its core, an SRAM cell is a small circuit that holds itself in one of two states. Two tiny switches support each other: one holds the other in its position, and vice versa. This state corresponds to a zero or a one. As long as voltage is applied, it persists on its own. That’s the “static” in static memory.

The comparison with DRAM makes the difference clear. There, a bit is stored as a small charge in a capacitor, similar to water in a leaky bucket. The charge leaks away, so the chip has to refill every value thousands of times per second. This refreshing costs time and energy. SRAM doesn’t need this and is therefore many times faster.

In practice, chip designers arrange memory in tiers, known as cache levels. Very close to the compute unit sits a very small, very fast SRAM area. Behind it follow larger, somewhat slower tiers. The search always proceeds from the inside out. If the chip finds a value already in the first tier, it saves an enormous amount of waiting time.

SRAM in graphics cards, phones, and financial news

You use SRAM every day without noticing it. It’s in the processor of your laptop, in the chip of your smartphone, and in every game console. When a manufacturer advertises “32 MB L3 cache,” they mean SRAM. Network devices like routers also use it to briefly buffer data packets.

In business news, the term comes up in discussions of AI accelerators. Some startups are building chips that aim to hold an entire model in SRAM, instead of loading it from slower memory. This promises very fast responses, but drastically limits model size. Other manufacturers instead rely on large amounts of fast stacked memory next to the compute chip.

A common misconception is that SRAM is simply “better RAM” that you could upgrade at will. That’s not possible: SRAM is fixed into the chip design and manufactured together with it. On top of that, there’s a problem currently occupying the industry. While compute units keep shrinking with new manufacturing processes, SRAM is barely getting any smaller. As a result, memory area is becoming an increasingly expensive part of modern AI chips.