ImageNet

ImageNet is a very large collection of photos from the internet. Each photo is labeled with what it shows: a golden retriever, a teapot, a school bus. These labels were assigned by hand by people, through painstaking work over years. In total, more than 14 million labeled images across around 20,000 categories were gathered this way. The collection was built starting in 2007 by a research team led by computer scientist Fei-Fei Li at Princeton and Stanford Universities. The purpose behind it: programs were meant to learn from these examples to recognize things in images on their own.

The competition that changed everything in 2012

Starting in 2010, there was an annual competition using ImageNet images, known as the ILSVRC. All participants were given the same subset of the collection: around 1.2 million images across 1000 categories. Their programs then had to guess what was shown in unfamiliar images. Whoever made the fewest mistakes won. Because everyone worked with the same material, the results were comparable directly for the first time.

In the early years, the winners' error rate was around 26 percent, and it declined only slowly. In 2012, a team from Toronto won with a program called AlexNet and an error rate of about 16 percent. This leap was so large that it upended the entire research direction. AlexNet was a neural network, meaning a program that derives its recognition rules entirely from examples itself, rather than having them specified by humans.

Afterward, almost all teams adopted this technique, and the error rate fell below five percent by 2015. This put the programs roughly at the level of a practiced human. Many experts date the beginning of today’s AI boom precisely to this competition. Without ImageNet as a shared benchmark, this progress would hardly have become visible.

How photos became training material

The images came from search engines, the labels from people. For this, the team used a platform where microtasks can be assigned to thousands of people on the internet for a few cents each. Around 49,000 people from nearly 170 countries took part. Each image was checked multiple times so that individual misjudgments would not slip through. The categories themselves came from WordNet, an existing database that organizes English-language terms by meaning.

During training, a program is shown one image after another, and its answer is compared to the correct label. With every mistake, the program’s internal settings are nudged slightly in the right direction. After millions of such rounds, it recognizes patterns that no one ever spelled out: edges, fur, typical shapes of ears. This is exactly why so many examples are needed. With a thousand images, this would never have worked.

Why ImageNet still underpins today’s products

A program trained on ImageNet has already learned to recognize general image features. This prior knowledge can be reused and redirected toward a new task with just a few hundred of one’s own images. Experts call this transfer learning. This gave rise to applications that classify skin lesions, inspect components in factories, or identify plant diseases from smartphone photos.

In the news, ImageNet usually appears as a benchmark: a new model is advertised by its accuracy on ImageNet. But the dataset also stands for a well-known problem. Some of the labels were faulty, some categories were offensive, and many photos predominantly showed Western everyday life. Anyone training on it inherits these biases. In 2019, more than 600,000 images were therefore removed from the collection.

Today’s image models often learn from billions of images along with automatically collected captions, without elaborate manual labor. This hasn’t made ImageNet obsolete, but rather shifted its role. It now serves as an established benchmark against which new methods can be measured against twelve years of history.