nDCG

When you search for something, you rarely get a single answer. You get a list: ten links, twenty products, five video suggestions. Such lists are ranked with varying degrees of quality. If the right match is in position 1, the list is helpful. If it’s in position 30, hardly anyone sees it. nDCG is a number between 0 and 1 that measures exactly that: how good a ranked result list is. The letter n stands for normalized, meaning the comparison to the best possible ranking.

Why the position of a hit matters

Simpler metrics only ask how many relevant results appear in the list. In practice, that’s not enough. Two search engines might both deliver five good hits. One places them in positions 1 through 5, the other in positions 16 through 20. For the user, these are completely different experiences. nDCG distinguishes between these cases because position is factored into the calculation.

On top of that: not every hit is equally good. When searching for a recipe, a detailed set of instructions is better than a forum post that merely mentions the dish. Both are somewhat relevant, but not equally strong. nDCG can handle such gradations. Humans or test data assign points for this, often ranging from 0 for useless to 3 for perfectly relevant.

That’s why nDCG is now the standard metric when companies want to improve their search or their recommendations. Changes to the ranking algorithm can be compared using it. If the nDCG value rises from 0.62 to 0.68, the new version is measurably better. Without such a number, developers would be left with nothing but gut feeling.

Collecting points with a discount going down

The calculation runs in three steps. First, every result in the list receives its relevance points. Then these points are devalued depending on their position: position 1 counts in full, position 2 counts a bit less, position 10 counts only a small fraction. This devaluation is called the discount. After that, all the discounted points are added up. This intermediate result is the DCG value.

The DCG value alone says little, because it depends on the task. For a search with many good hits, it automatically comes out high; for a difficult search, it comes out low. That’s why the final step follows: you calculate what DCG value the perfect ranking would have achieved. Perfect here means all hits ordered by relevance in descending order. Dividing the actual value by this ideal value yields the nDCG.

As a result, the outcome always lies between 0 and 1. A value of 1 means: it couldn’t have been ranked any better. Usually only the first positions are considered, and this is noted accordingly, for example nDCG@10 for the first ten results. One important point is often overlooked: nDCG only measures the ordering. If a good document is missing from the database entirely, the metric doesn’t catch that.

From Google search to AI chatbot

Search engines, online stores, and streaming services evaluate their ranking using nDCG. Amazon wants relevant products at the top, Spotify wants relevant songs. In all these cases, the order determines what users even get to see. Studies show that the first three hits receive the vast majority of clicks.

The metric also comes up with AI chatbots. Many systems search for matching text passages in a document collection before generating an answer. This method is called Retrieval-Augmented Generation. Only the top-ranked findings make it into the model, because the space for input text is limited. If the crucial document is at rank 12, it’s useless. That’s why the retrieval component of such systems is regularly evaluated using nDCG.

In technical articles and benchmark tables, you’ll come across figures like nDCG@10 on the BEIR dataset. This refers to a standardized benchmark test for search models. Anyone reading such numbers should pay attention to two things: what data was used for the measurement, and up to which rank. An nDCG@100 is almost always higher than an nDCG@10, and therefore not directly comparable.