LPU
An LPU (Language Processing Unit) is a specialized chip built exclusively to make finished language AI models respond quickly. The term became known through the US company Groq, whose chips output text noticeably faster than typical graphics cards.
Programs that write text or answer questions need a great deal of computing power. This computational work is normally handled by graphics cards, i.e. chips that were originally developed for video games. An LPU is a different approach: a chip built from the ground up for a single task, namely outputting text word by word. The abbreviation stands for Language Processing Unit. The name comes from the US company Groq, which has made such chips known since 2024. The advantage shows up especially in speed: answers often appear faster than a human can read along.
Why speed in text output becomes a business model
Large language models don’t generate text all at once. They compute one word fragment after another, strictly in sequence. Each new word fragment needs the entire text generated so far as its basis. That’s why this work is hard to spread across many computing cores, and that is exactly where graphics cards are actually strong.
For users, speed is not a luxury but decides usability. A chatbot that takes ten seconds for an answer feels sluggish. This becomes even more important with AI systems that query themselves multiple times before answering. There, waiting times quickly add up to minutes.
Then there’s the money. Anyone answering millions of requests per day pays for electricity and chip time for each one. Hardware that does the same work with less energy permanently lowers ongoing costs. That’s why providers like Groq, Cerebras, or SambaNova are investing billions in their own chips, even though Nvidia dominates the market.
What an LPU builds differently than a graphics card
The most important difference lies in memory. Graphics cards have large but comparatively slow memory next to the compute chip. For every single word fragment, the model’s numbers must be fetched from there. So the chip often waits instead of computing. Experts call this a memory bottleneck.
An LPU reverses this. The memory sits directly on the chip, right next to the compute units. This is extremely fast, but very small: only a few hundred megabytes per chip. A large language model doesn’t fit in there. That’s why hundreds of LPUs are linked together and the model is distributed across them in slices.
Added to this is fixed scheduling. With an LPU, it’s already determined before the start which chip performs which calculation in which fraction of a second. There is no spontaneous distribution during operation. This makes execution very even and predictable. A common misconception, by the way, is that LPUs train AI models. They don’t — they are pure answer machines for already fully trained models.
LPUs in headlines and in finished products
The term is most often encountered in reports about Groq. The company was founded by a former Google engineer who had worked there on AI chips. Groq advertises figures like several hundred word fragments output per second with open models such as Llama. For comparison: a human reads roughly five to ten word fragments per second.
Private individuals cannot buy such chips directly. You encounter them indirectly when an app or a chatbot has a particularly snappy AI built in. Developers book the computing power over the internet and pay per amount of text processed.
Economically, the term is interesting because it stands for the attempt to challenge Nvidia’s dominance. Nvidia makes most of its money with chips for training. LPUs target the other, steadily growing part of the market: the daily operation of finished models. Whether this attack succeeds remains open — and that’s exactly why “LPU” regularly appears in financial news.