Classifier

A classifier is a program that assigns a thing to one of several predefined pigeonholes. The pigeonholes are fixed in advance: “spam” or “not spam,” “cat,” “dog,” or “bird,” “invoice” or “reminder.” The classifier is presented with something — a text, an image, a series of measurements — and names the matching pigeonhole. Importantly: it may not invent a new pigeonhole. It always chooses only from the categories it has been given. If it encounters an image of a bicycle even though it only knows animals, it will still answer with an animal.

Why sorting solves so many problems

Surprisingly many practical tasks are, at their core, sorting tasks. Is this transfer fraud or not? Is this skin spot harmless or suspicious? Is this comment an insult? In all these cases, an answer from a short list is enough. That is exactly what a classifier is built for.

The advantage lies in scale. A human can perhaps check a few hundred emails a day. A classifier checks millions without ever getting tired. That is why such programs now sit everywhere too many cases pile up for too few humans.

However, classification should be distinguished from a related task. If the result is not a category but a number on a scale — say, an estimated apartment price — this is called regression. Classification delivers pigeonholes, regression delivers numbers. This distinction appears in almost every text on machine learning.

From a stack of examples to a decision boundary

Almost all classifiers learn from examples. One collects many cases for which the correct category is already known. For spam, that would be about 50,000 emails that humans have sorted by hand. This stack is called training data. The program searches within it for features that are typical of a category.

Put visually, the classifier draws a dividing line. Imagine all emails as points on a plane: on the right, those with many exclamation marks and sums of money; on the left, the normal ones. Training shifts the line until it separates the two groups as well as possible. A new email then lands on one side of the line — and receives that side’s category.

Internally, a classifier usually does not give a hard answer but a confidence level: “87 percent spam.” Only a fixed threshold turns this into a verdict. This threshold can be shifted, and that has consequences. Set it strictly, and more spam slips through. Set it loosely, and important emails end up in the spam folder. Both types of error can never be reduced to zero at the same time.

Classifiers in the inbox, the clinic, and the chatbot

The spam filter is the best-known example and works for you every day. The photo gallery on your phone, which sorts images by people, animals, and landscapes, works similarly. Banks also use classifiers to block suspicious card payments. In medicine, they assess X-ray and skin images as a second opinion for doctors.

In the news, classifiers are often mentioned in connection with criticism. If an application filter rates women worse, this is usually due to the training data: it contained past decisions, complete with biases. The program simply adopted these patterns. Such cases show that a classifier is no more objective than the examples it learned from.

Modern chatbots also contain classifiers, only invisibly. A small auxiliary model there checks, for instance, whether a request violates the provider’s rules. Another decides whether the question is simple or requires a larger model. The term thus does not denote a specific technique but a type of task — it can be handled by a few computational rules or by a large neural network.