Recurrent Looped Transformer
A Recurrent Looped Transformer is an AI model that passes data through the same computational building blocks multiple times in a row, instead of using new ones for each step. This allows it to "think through" difficult tasks for longer without the model growing in size.
Language models like ChatGPT consist of many similar computational layers. A piece of text flows from bottom to top through these layers, with each one refining the intermediate results a bit further. Normally, each layer has its own learned settings, and each is passed through exactly once. A Recurrent Looped Transformer breaks with this rule: it takes a small group of layers and sends the data through that same group multiple times. The loop can run for two rounds or twenty. This makes the model compute for longer, but it stays the same size, since no new settings are added.
Thinking time instead of model size
For years, a simple rule applied in AI development: more capability comes from more size. Developers stacked more layers and stored more learned numbers, the so-called parameters. But this drives up costs. A model with twice as many parameters needs twice as much memory on the expensive graphics cards that power such systems.
The loop shifts this trade-off. A model with six layers that runs four rounds computes roughly as much as a model with 24 layers. But it only occupies the memory of six. For devices with limited memory, such as smartphones, this is a genuine advantage.
A second point is especially interesting: the number of rounds can still be changed after training. An easy question gets two rounds, a math problem gets twelve. The model can thus adapt its effort to the difficulty of the task. In research, this principle is called adaptive computation depth.
What happens in the loop
One can imagine the process like revising an essay multiple times. The first pass produces a rough draft. On the second pass, the same person reads the text again and makes improvements. It is not a new editor each time, but always the same one – just with a better text in front of them.
Technically, this means the output of the last layer in the block becomes the input to the first layer of the same block again. So that the model doesn’t get confused, it is usually given a counter indicating the current round number. This allows it to work differently in early rounds than in later ones. The whole thing is trained with a varying number of rounds, so that the model can cope with many different loop depths.
The price is computation time. Twelve rounds take twelve times as long as one. What is saved is memory, not time. There is also a limit: beyond a certain number of rounds, the result barely improves anymore, or even gets worse, because the intermediate results start to compound. A common misconception is therefore that more loops are always better.
Related ideas and current models
The term appears mainly in research papers and specialist news, less often in marketing for finished products. A well-known example is Google's model family with shared layers, as well as the research model Huginn, which explicitly describes its loops as “latent thinking.” Several Chinese labs also released variants in 2024 and 2025.
This should be distinguished from so-called reasoning models. They also think for longer, but they write down their intermediate steps as visible text. A Looped Transformer, by contrast, thinks internally, in numbers, without words. The two approaches can be combined, but they are not the same thing.
Economically, the approach is especially interesting for on-device AI. Anyone who wants to run a model directly on a phone struggles with limited memory but often has some patience to spare. This is exactly the trade-off the loop is suited for. When reports mention “small models with big performance,” this technique is sometimes behind it.