Binary Noise

In technology, noise refers to any random disturbance that overlays a signal. On the radio you hear it as crackling; in old photos you see it as graininess. Usually this disturbance is continuous: a value becomes a little too high or a little too low. Binary noise works differently. It knows only two states, such as 0 and 1 or black and white. Each disturbed point therefore flips completely instead of merely shifting slightly. A well-known example is an image in which individual pixels suddenly turn pure white or pure black.

Why exactly two states cause so much trouble

Computers store everything as a sequence of zeros and ones. If a single one of these characters flips during transmission, exactly this kind of disturbance occurs. The resulting damage is hard to predict. If an unimportant character flips, nobody notices. If an important one flips, an entire file can become unreadable.

This distinguishes binary noise from the soft variant, so-called Gaussian noise, in which all values are only slightly blurred. Soft disturbances can often be averaged out by combining neighboring measurements. With hard flips, this works poorly. A single white pixel in the middle of a dark area strongly distorts the average of its neighbors. That is why different filters are needed—ones that detect an outlier and replace it rather than factoring it into the calculation.

In practice, this means: systems that have to deal with binary noise need built-in safeguards. Without them, no radio connection and no hard drive would be reliable enough for serious applications.

How the disturbance arises and how to get rid of it again

The causes are physical. In a radio signal, electromagnetic disturbances overlap; a memory chip loses its charge due to heat or aging; a camera sensor has defective pixels. In all cases, the electronics ultimately decide between two possibilities and sometimes decide wrongly. This can be described with a single number: the probability that an individual character flips. If it is one percent, on average one in a hundred characters is wrong.

Such errors can be countered with verification methods that send along additional information. In the simplest case, each message is sent three times, and in the end the version that occurs twice is used. Modern methods are more economical and manage not just to detect errors but to correct them directly. For images, on the other hand, the median filter is often used: it replaces each point with the median value of its neighbors, causing individual outliers to disappear.

In AI, the tables are turned and the disturbance is generated deliberately. During training, a model is deliberately shown damaged data and is still required to give the correct answer. This forces it not to rely on individual details. A related trick is called dropout: during training, parts of the network are randomly switched off, likewise following the on-or-off principle.

From the scratched disc to the training run

In everyday life, this phenomenon is encountered constantly without ever getting a name. A scratched DVD shows blocky artifacts in the picture, poor radio reception makes voice messages stutter, an old USB stick suddenly delivers a corrupted file. Behind each of these are flipped zeros and ones. The fact that this rarely happens is thanks to the correction procedures working in the background.

In trade news, the term mainly appears in two contexts. First, with quantum computers, whose computing units are extremely susceptible to disturbance and therefore require elaborate error correction. Second, in the training of AI models, where artificially added disturbances are considered a method for avoiding overfitting. Overfitting means that a model memorizes its training examples and fails on new data.

A common misconception is to confuse binary noise with the noise in image-generation models. There, an image is gradually dissolved into uniform graininess and then reconstructed. However, this graininess is soft and continuous, not binary. Both procedures use randomness but follow different rules.