Random Forest

A random forest is a method that allows a computer to derive predictions from existing data. Its foundation is a so-called decision tree: a chain of yes-no questions that ultimately leads to a result. Such a chain might go: Is the customer older than 40? Has he ever canceled before? At the end stands an assessment such as “will likely remain a customer.” A single tree, however, is often off the mark, because it latches on too strongly to the randomness of its training data. A random forest therefore builds hundreds of such trees in slightly different ways and lets them vote together. The English word “forest” describes a wood, “random” means arbitrary or by chance – the name thus describes exactly the principle at work.

Why many trees guess better than one

Behind this lies an idea familiar from everyday life. If a single person estimates the number of jellybeans in a jar, they’re usually significantly off. Ask a hundred people and average their answers, and the result is astonishingly close to the truth. Estimates that are too high and too low cancel each other out. This exact effect also works with decision trees.

The technical term for the problem of a single tree is overfitting. This means that a model memorizes the training data instead of recognizing general patterns. It then fails on new data. A forest made up of many different trees is considerably more robust against this error, because no single tree alone determines the outcome.

On top of that comes a practical advantage: random forests often work well even without elaborate fine-tuning. On tabular data, meaning classic tables with rows and columns, they frequently even outperform large neural networks. That’s why they remain in use today at banks, insurance companies, and in industry, even though the method dates back to 2001.

Randomness in two places

To keep the trees from all turning out the same, randomness is deliberately introduced at two points. First, each tree doesn’t receive the complete dataset, but rather a randomly drawn sample from it. Some data points appear in it multiple times, others not at all. This sampling with replacement is called bagging.

Second, at each split a tree is not allowed to examine all features, but only a random selection of them. If a dataset has twenty columns, perhaps only five are available for choice at any given split. This prevents one particularly conspicuous feature from ending up at the top of every single tree. This is how genuinely different trees emerge, rather than a hundred copies of the same one.

When making a prediction, all the trees are then consulted. If it’s a yes-no decision, the majority vote wins. If it’s a numerical value, such as a property price, the average of all trees is calculated. A common misconception is that more trees eventually become harmful. That is not the case: beyond a few hundred trees, the result barely improves further, but it doesn’t get worse either.

From loan applications to disease diagnosis

Random forests are embedded in many systems that people never see directly. Banks use them to estimate how likely it is that a loan will be repaid. Online shops predict which customers might churn. In medicine, they help draw indications of diseases from lab values. They are also widespread in fraud detection for card payments.

In news coverage about artificial intelligence, the term appears less often than language models or neural networks. This is because random forests are unspectacular and have been tried and tested for a long time. For images, speech, or text they are unsuitable, and there neural networks are clearly superior. Their strength lies with structured numerical and categorical data.

Another reason for their popularity is interpretability. A random forest can indicate which features were particularly important for its decisions. This is often a legal requirement in loan approvals or medical applications. A forest of 500 trees is admittedly not fully transparent, but it is considerably more transparent than a large neural network.