SOCAMM

Every computer needs working memory. That’s the fast buffer where data resides while the machine works with it. In normal PCs, this memory sits on small circuit boards that can be pulled out and swapped. SOCAMM is a new form factor for such plug-in modules, developed for the large machines that run artificial intelligence. The abbreviation stands for Small Outline Compression Attached Memory Module. What’s special about it: these modules carry memory chips of the kind otherwise known from smartphones, because they consume very little power.

Why Nvidia needed its own memory form factor

AI data centers have a power problem. A single large facility can consume as much energy as a small town. A noticeable share of that goes not into the compute chips but into the working memory. Conventional server memory modules are built for continuous load but not optimized for efficiency. Smartphone memory is designed exactly the other way around, because there every milliwatt strains the battery.

Until now, however, there was a catch. This power-saving memory could only be permanently soldered onto the mainboard. If a single chip fails, the entire expensive board has to be replaced. Later upgrades are also impossible then. In installations with tens of thousands of servers, that’s a real cost problem.

SOCAMM resolves exactly this trade-off. The memory remains power-efficient but once again sits on a swappable module. Nvidia states that the modules draw significantly less power than classic server modules. For memory manufacturers like Samsung, SK Hynix, and Micron, this is an important new market, because they can now sell their smartphone chip technology into data centers as well.

The design of the module

A SOCAMM module is roughly the size of a matchbox. Several memory chips of the LPDDR type are mounted on it. The L stands for low power, meaning low power consumption. This type of chip is found in almost every smartphone today.

The module is not connected via a long contact strip along the edge, as with a PC memory module. Instead, it lies flat and is fastened to the board with three screws. This makes it very low-profile, which matters in densely packed server chassis. The short connection paths also allow high data rates to be achieved. The second generation, SOCAMM2, is meant to significantly increase transfer speed once again.

It’s important to distinguish this from HBM. That’s a different memory type that sits directly next to the compute chip and is far faster still, but also extremely expensive. HBM handles the time-critical computational data. SOCAMM, by contrast, is the large, cheaper main memory alongside it. So the two types don’t replace each other; they work side by side in the same machine.

SOCAMM in Nvidia’s product line and in stock market news

In everyday life, you’ll hardly ever hold a SOCAMM module in your hand. It sits in server racks located in data centers. Among other things, it is used in Nvidia’s Grace-Blackwell family of systems, i.e., the platforms on which large language models are trained. The small developer computer DGX Spark also uses this design.

Nevertheless, the term regularly appears in business news. Reports about supply agreements between Nvidia and memory manufacturers move their stock prices. For Samsung, SK Hynix, and Micron, this involves orders worth billions. Anyone reading such reports should know that behind the unwieldy acronym is simply a memory module.

A common misconception is that SOCAMM makes computers faster. The actual gain lies in power consumption, build height, and serviceability. Whether the form factor will establish itself as a standard in the long run remains open. So far, it is being pushed mainly by a single manufacturer, and official industry standards for server memory normally emerge from committees with many participants.