MGREP

MGREP is a computer program that searches large amounts of text for terms from a predefined list. This list can, for example, contain several hundred thousand medical technical terms. The program reads through the text and marks every spot where one of these words occurs. The name alludes to an old tool called grep, which is used to search for text passages on computers. The added M indicates that MGREP does not process just one search term, but many at once. It was developed at the University of Michigan, and from 2008 onward it became the core of a well-known search engine for medical terminology.

Why technical terms in texts need to be marked

Scientific texts are readable for humans, but for computers they are initially just a string of letters. A program does not know by itself that a sentence mentions a disease, a medication, and a body part. That is exactly what MGREP does: it turns running text into a list of recognized technical terms. Only after that can the text be meaningfully searched, sorted, or analyzed.

The need is enormous. In medicine alone, more than a million technical articles appear every year. No one can read them all. But if a machine can recognize which diseases and active substances are being discussed in every article, connections can be found across the entire body of literature. The same applies to hospitals, which have millions of physician letters on file.

Speed is crucial here. Methods that grammatically parse every sentence are thorough but slow. MGREP was deliberately tuned for speed, allowing it to process very large text collections in a reasonable amount of time. It is precisely this trade-off that has made it so popular in practice.

Radix trees and the trick with word stems

The naive solution would be to search the text separately for each term on the list. With 300,000 terms and millions of texts, that would take forever. MGREP therefore turns the task around. It builds a kind of huge directory from the term list, in which words with the same beginning share the same path. Experts call such a structure a radix tree. After that, the text only needs to be scanned once.

A second point is tolerance toward spelling variations. People write “heart failure” sometimes in singular, sometimes in plural, sometimes with a hyphen, sometimes in a different word order. MGREP reduces words to their base form and ignores small deviations. This way it also finds variants that are not literally listed.

The limits lie in comprehension. MGREP recognizes character strings, not meaning. If a text says “no indication of diabetes,” the program still flags the word diabetes. Ambiguous abbreviations like “MS” also remain a problem. That is why such tools are nowadays usually followed by additional programs that check the context. Modern language models, i.e. AI systems that process texts in terms of content, are clearly better at this, but also considerably slower and more expensive.

MGREP in research databases and clinical software

MGREP is most commonly encountered in the BioPortal of the US research consortium NCBO. There you can insert any text and have all the technical terms it contains displayed, along with references to the matching catalog entries. This service is used by universities and pharmaceutical companies to automatically tag studies and datasets with keywords.

In everyday life you encounter this principle indirectly. When a health app extracts diagnoses from a physician’s letter, or a literature search suggests suitable articles to you, a term recognizer like this is often behind it. In companies, the same technology can also be found outside of medicine, for instance for product names in customer emails.

In technical news and specialist articles, MGREP usually appears in comparisons. It is considered a fast but simple method and serves as a benchmark for newer systems. A common misconception is to mistake MGREP for an AI. It learns nothing and makes no decisions – it merely searches very cleverly.