Annotation

Annotation means: humans add extra information to data. On a photo, someone draws a box around a bicycle and labels it “bicycle.” With a text, someone marks whether a customer review sounds friendly or angry. With an audio recording, someone transcribes what was said. These additional pieces of information are called labels. They are the correct answer against which a computer program can measure its output.

Why models remain blind without labels

A learning program needs feedback. It first guesses, then compares its result with the label, and corrects itself. Without a label, there is nothing to compare against. Raw data is therefore cheap, but on its own often worthless. The actual value only emerges through the labeling.

That’s why annotation is a major cost factor. Taking a photo takes a second, but marking it up cleanly takes several minutes. For medical images, even a doctor has to do it, and the hourly rate is priced accordingly. Large companies employ thousands of people for this, often in countries with low wages. The working conditions of these workers are regularly a topic in media reports.

Quality beats quantity here. Incorrect labels systematically teach the model something wrong. A well-known example: if snow appears in all the annotated wolf images, the model might learn “snow” instead of “wolf.” Such errors only become apparent once the finished system fails in practice.

From raw image to labeled dataset

At the start there is a set of rules, the annotation guideline. It defines exactly what is to be marked. Does a scooter count as a motorcycle? Do the wheels belong inside the bounding box? Without such specifications, every worker labels differently, and the result becomes unusable.

After that, the annotators work in specialized software. They draw boxes, paint outlines, or select categories from a list. Part of the data is deliberately processed twice. If two people rarely agree, the guideline is usually unclear. This agreement is measured as a metric and serves as quality control.

To save time, an existing model is often used to do preliminary work. It makes a suggestion, and the human only confirms or corrects it. A related approach is called active learning: the model reports which cases it is particularly uncertain about, and exactly those are labeled by hand. Both approaches significantly reduce costs but do not replace humans.

Annotation in chatbots, cars, and job postings

In language models like ChatGPT, annotation is present in a special form. Test subjects receive several draft answers and rank them by quality. From these ratings, the system learns which tone is desired. Experts call this method RLHF, i.e., learning from human feedback. Even the decision about which content counts as dangerous is, in the end, an annotation decision made by humans.

On the road, the connection is more visible. For driver assistance systems, people mark pedestrians, traffic lights, and lane edges on millions of video frames. Something similar happens in medicine, where tumors are outlined on X-ray images. Spam filters and translation services are also based on labeled examples.

Even people who don’t work in the industry regularly help with annotation. Every solved image puzzle at login provides labels. So does every thumbs-up under an AI answer. In job postings, the profession appears as “Data Annotator,” and publicly traded service providers in this field are treated as a distinct market in business news.