PRISMA

For many questions there isn’t just one study, but hundreds. Some researchers therefore collect all existing studies on a topic and evaluate them together. Such compilation works are called systematic reviews. PRISMA is a set of rules for how such a work must be described so that others can follow it. The abbreviation stands for “Preferred Reporting Items for Systematic Reviews and Meta-Analyses,” meaning roughly: recommended information for reviews. PRISMA does not prescribe how research is conducted, but what must be stated in the text at the end.

Why reviews without fixed rules are dangerous

A review appears especially trustworthy. After all, it summarizes not just one experiment, but many. Precisely for this reason it is easy to manipulate. Anyone who only mentions the studies that fit the desired result creates a false overall picture. Without rules, this is hardly recognizable from the outside.

PRISMA forces the authors to disclose their approach. They must state which databases they searched, with which search terms, and on what date. They must state how many hits they excluded and why. This makes it possible to check whether inconvenient studies have disappeared.

For many medical journals, PRISMA is now mandatory. Anyone submitting a review must include the completed checklist. Authorities that decide on medications or vaccination recommendations also rely on such works. A report that does not meet PRISMA standards is often not taken seriously there at all.

Checklist and flow diagram in detail

PRISMA consists of two parts. The first is a checklist. In the current 2020 version, it has 27 items, spread across title, abstract, methods, results, and discussion. Each item describes a piece of information that must appear in the text. Example: item 7 requires the complete search query for at least one database, word for word.

The second part is the flow diagram. It shows the path of the studies from top to bottom, stage by stage. At the very top is the number of hits from the database search, say 4,312. Below that, duplicates are removed, i.e., studies that appear in multiple databases. Then titles and abstracts are screened, and later the full texts are read. In the end, perhaps 18 studies remain that are actually evaluated.

What matters is that at each stage the number of excluded works and the reason for exclusion is given. Typical reasons are: wrong age group, no control group, text not retrievable. The diagram works like a sieve with several layers, where you can recount at each layer. A common misconception is that PRISMA is a quality seal for the studies themselves. It is not. It only guarantees transparency about the process, not the quality of the individual studies.

PRISMA in AI research and in the news

PRISMA originally comes from medicine. Meanwhile, psychology, education research, and increasingly computer science also use it. When someone examines how well language models perform on medical exam questions, there are dozens of individual studies on the topic. A review bundles them, and PRISMA governs their structure. In such works, you will usually find the flow diagram as the first figure.

In news articles, you rarely encounter the term directly. But phrasing like “an analysis of 40 studies shows” almost always refers to such a work. Anyone who wants to check whether the selection was fair looks for the PRISMA diagram in the original publication. The numbers there reveal more about reliability than the article’s headline.

You should not confuse PRISMA with products of the same name. There is software, data projects, and even surveillance programs with similar names. In a scientific context, it practically always refers to the reporting guideline. Alongside it, related guidelines exist for other types of studies, such as CONSORT for clinical trials.