Collaborative Training
In collaborative training, multiple parties work together on an AI model even though each keeps their own data or computers. Instead of gathering everything in one place, only intermediate results are exchanged and merged into a shared model.
A computer program that learns from examples is called a model. For it to learn something, it needs a great many examples and a great deal of computing power. Normally, both are brought together in one place: all the data on one large computer cluster, where the computation then takes place. In collaborative training, this is deliberately done differently. Several hospitals, companies, universities, or even private individuals with their own computers work together on a model without handing over their data or devices. Everyone computes locally, and only the learning results are exchanged.
Why data often cannot be pooled together
Many of the most valuable datasets simply cannot be sent elsewhere. A hospital’s patient records fall under data protection law. A car manufacturer will not hand its test-drive data over to a competitor. A bank is not allowed to copy account transactions onto foreign servers. Without collaborative training, this knowledge would go unused, because no single participant has enough examples to build a good model on their own.
The second reason is computing power. Training large models costs tens of millions and requires thousands of specialized graphics chips. Only a handful of corporations worldwide own these chips. When many small participants link their computers together, a kind of shared supercomputer emerges. This is exactly what several open-source projects are attempting, in order not to be dependent on the large providers.
A third point is often overlooked: diversity. A model that learns only from data at one clinic knows only that clinic’s patients and equipment. Twenty clinics from different countries together produce a significantly more robust model. Collaborative training is therefore not merely a compromise, but often leads to better results.
The cycle of computing and merging
The process is a repeating cycle. A server first distributes the current state of the model to all participants. Each participant continues training this copy using their own local data. In doing so, the model’s internal numerical values, called parameters, change. Only this change is sent back, not a single data example.
The server then averages all the incoming changes into a new shared model. This is sent back to everyone, and the next round begins. One can picture it like a study group in which each person works through a different chapter alone. Afterward, everyone meets and compares notes. No one has to hand over their book.
However, the method is certainly not automatically fully secure. It may be possible to draw conclusions about the original data from the changes sent back. For this reason, collaborative training is often combined with additional techniques that deliberately mix noise into the values. A second problem is the connection: if participants communicate over ordinary internet lines instead of fast data-center networks, sending the parameters quickly becomes a bottleneck.
From the smartphone keyboard to research alliances
The best-known example is found in almost every smartphone. The keyboard’s word suggestions learn which phrasings are typical. Part of this learning happens directly on the device, so that private messages never leave the phone. Google made this method famous under the name Federated Learning, which is the most widespread variant of collaborative training.
In medicine, clinics join together in research consortia to, for instance, train tumor detection on X-ray images. The term also appears in the news when open-source projects train language models in a distributed fashion over the internet. Such projects see themselves as a counterweight to the closed models of individual corporations. Anyone reading the term in a news report should check whether data truly remains separate or whether only computing load is being distributed. Both are sometimes sold under the same label.