Adapter (Parameter-Efficient Fine-Tuning)
Adapters are small additional building blocks inserted into an already trained AI model to adapt it to a new task. Instead of retraining the entire model, only these few building blocks are changed – saving computing time, memory, and money.
Large language programs like ChatGPT consist of billions of adjustable numbers. These numbers are called parameters, and they contain everything the program has absorbed during learning. If you want to switch such a finished program to a specialized task, for example legal texts, you could readjust all the numbers. But that is extremely expensive and requires powerful computers. An adapter is the frugal way: you freeze the existing numbers and add only a few new ones. Only these new numbers are changed, often less than one percent of the entire model.
What adapters save in cost and memory
Completely retraining a model requires graphics cards worth several hundred thousand euros. A single good graphics card is often enough for an adapter, sometimes even a rented one for a few euros per hour. This allows even small companies, universities, or individuals to tailor a large model to their purpose. This is one of the reasons why there are so many specialized AI tools.
The second advantage concerns storage. A complete customized copy of a large model quickly takes up several hundred gigabytes. An adapter for this purpose is often only a few megabytes in size. A provider can therefore keep a base model once on its servers and place hundreds of adapters for hundreds of customers alongside it. When answering a request, only the matching adapter is then switched in.
There is also a security aspect that is easily overlooked. Because the base model remains unchanged, it cannot be accidentally damaged. If an adapter produces poor results, you delete it and start over. With full retraining, on the other hand, a model sometimes loses abilities it previously had.
Small intermediate layers instead of new billions
An AI model is built up in layers that data passes through in sequence. In the classic adapter method, tiny additional layers are inserted between these layers. They first shrink the data down to a few values, compute on them, and then inflate them back to their original size. This detour through a bottleneck is the trick: it ensures that only very few new numbers are needed.
During training, the model is shown what it should do better. However, the correction flows exclusively into the additional layers. All original numbers remain untouched. You can think of it like a pair of glasses: the eye stays as it is, the glasses only redirect the light a little.
The most common variant today is called LoRA. It does not insert its own layer, but places a small correction table alongside the existing numbers. Its advantage: after training, this correction can be merged with the original, so it costs no additional computing time during use. The umbrella term for all these methods is parameter-efficient fine-tuning.
Adapters in apps, image generators, and corporate AI
Adapters are most visible in image generators. Online collections contain thousands of small files that teach a generator a particular drawing style, a person, or an object. Each of these files is an adapter, usually a LoRA file of a few megabytes. You load it in addition to the base model and switch it on or off as needed.
In companies, the same technique is used to trim a model to their own specialized jargon. An insurance company can thus adjust a general language model to its contract wording. Cloud providers such as Amazon, Google, or Microsoft offer this process as a service, often for a few euros per training run.
A common misconception: adapters do not give the model new knowledge in the sense of facts. They primarily shift the behavior, style, and format of the answers. Anyone who needs current data is better off combining the model with a search of their own documents. Adapters and search are not mutually exclusive but are in practice often used together.
