Regression (Machine Learning)

Regression refers to a particular kind of task that a computer program can learn from data. The program is given many examples with known answers and is supposed to predict a number from them. A typical example is the price of an apartment: known are size, year of construction, location, and floor; sought is the price in euros. The result is a value on a continuous scale, i.e. 412,350 euros and not merely “expensive” or “cheap”. This is exactly the difference to so-called classification, where a program chooses between fixed categories, such as “spam” or “not spam”. Regression is thus one of the oldest and most widely used tools of data analysis altogether.

Why predicting numbers is worth so much

Very many practical questions come down to a number. How much electricity will a city consume tomorrow at noon? How long will delivery take? How high will revenue be next quarter? For all of this, a category is too coarse; a concrete value is needed. Companies plan their purchasing, staffing, and pricing based on such predictions.

A second reason is traceability. Simple regression models can be read like a formula. One can say: Each additional square meter increases the estimated price by around 4,000 euros. Such statements are important in areas like medicine, insurance, or lending, because decisions must be justified there.

A common misconception, however, is that a model thereby recognizes causes. It only finds correlations in the data. If ice cream sales and sunburns rise together, one does not cause the other. Both depend on the weather. Regression predicts well but does not automatically explain why something happens.

From the trend line to the error measure

The simplest form is linear regression. One plots the example data on a coordinate system and draws a straight line through it. The line is shifted and tilted so that it lies as close as possible to all the points. Afterward, for each new input value, one can read off the corresponding point on the line. That is the prediction.

What “as close as possible” means must be measured. For this, one calculates the distance of each point from the line, squares it, and adds everything up. This sum is called the error function, and training consists of making it small. Squaring ensures that large outliers weigh heavily. A method called gradient descent feels its way in small steps toward the best setting.

In practice, there are usually many input variables instead of one, and the relationship is rarely a pure straight line. In that case, one uses decision trees, tree ensembles, or neural networks. The principle remains the same: the output is a number, and an error measure evaluates how far off it is. The only real danger is overfitting, when a model memorizes the training examples and fails on new data.

Regression in weather apps, stock market reports, and delivery times

In everyday life, regression is embedded in many displays without the term ever being mentioned. The arrival time in a navigation app is a regression prediction. So is the temperature forecast for the day after tomorrow, the remaining battery life indicator, or the estimated resale value of a car on a used-car website.

In economic news, the term often appears in the form “model forecasts”. Banks estimate default risks, energy providers estimate grid load, retail chains estimate sales of individual products. Figures on inflation or economic growth are also frequently based on regression models using historical data.

Anyone reading such figures should pay attention to two things. First, the uncertainty: serious forecasts state a range, not just a single point value. Second, the underlying data, because a model only knows the past. In the event of sudden upheavals such as a pandemic or a price shock, regression predictions regularly turn out to be significantly off.