Federated Learning

Programs that learn from examples usually need very large amounts of data in one place. Typically, a company collects this data on its own machines and analyzes it there. Federated Learning turns this around: the data stays where it originates, for example on your phone or in a hospital. Instead, a copy of the program is sent to the devices. Each device learns on its own from its own data and sends back only what it has learned. The central server combines all these responses into an improved version without ever having seen a single photo, message, or patient record.

Privacy as a design principle

Many of the world’s most interesting data sets simply cannot be collected. Medical records, bank transactions, private chat histories: strict rules apply to all of these in Europe. A clinic is not allowed to simply send its patient data to a tech company. Until now, this ruled out exactly the areas where better predictions would be most useful.

Federated Learning at least partially resolves this conflict. Ten clinics can jointly train a model for tumor detection without sharing a single scan. Each clinic benefits from the experience of the other nine. Legally, data sovereignty still remains in-house. This argument is the main reason the method has attracted so much attention since around 2017.

A second advantage is practical in nature: sending data costs bandwidth and time. Billions of phones generate text data every day that nobody could ever fully upload. A few thousand numbers as a learning result, by contrast, are tiny. Federated Learning is therefore not only more discreet, but often simply cheaper.

How the rounds between server and devices work

Training proceeds in rounds. The server sends the current version of the model to a selection of devices, often just a few thousand out of many millions. Each device locally computes a few learning steps using its own data. Afterward, it does not send back the data, but rather a list of update values. These values describe how the model would need to be adjusted.

The server forms an average from all the responses and uses it to update the shared model. This procedure is called Federated Averaging and is the best-known variant. Then the next round begins with a new selection of devices. After hundreds or thousands of such rounds, the model is finished.

A common misconception is that the update values are automatically anonymous. Under certain circumstances, it is possible to reverse-engineer what was on a device from them. That’s why the method is usually combined with two protective measures. With secure aggregation, the server can only decrypt the sum of all responses, never an individual one. With differential privacy, random noise is deliberately added to make individual contributions unrecognizable.

Keyboard, voice assistant, and hospital network

The best-known example is found in Android phones. The Gboard keyboard learns new words and better word suggestions directly on users' devices. When a buzzword suddenly appears everywhere, the keyboard picks it up after a short time, without Google having collected typing histories. Improvements to voice assistants and on-device photo search work in a similar way.

In the news, you’ll mostly encounter this term in connection with medicine and finance. Research consortia train diagnostic models across national borders. Banks jointly search for fraud patterns without exchanging account data. Automakers also use the principle so that driver assistance systems can learn from rare traffic situations.

It’s important to draw a distinction here: Federated Learning is not a replacement for central data centers. Large language models are still built the classic way, on huge server farms. The method pays off primarily where data is distributed, sensitive, and available in massive quantities. And it has limits: phones are slow, go offline, and have highly varied data. This makes training more unstable and slower than a centralized solution.