Stylometry

Every person writes differently, without noticing it. One person uses many commas, another writes short sentences. Some constantly reach for words like “however” or “actually,” others almost never. Stylometry makes exactly such habits countable. It breaks a text down into measurable features and compares the numbers with those of other texts. The goal is usually one question: Does this text come from the same person as that other one?

When there’s no name under the text

Anonymous and disputed texts exist everywhere. In literary studies, people have argued for centuries over whether Shakespeare wrote all his plays himself. With old religious and historical writings, the author is often unknown. Stylometry provides an argument here that is not based on gut feeling, but on frequencies.

The method is also in demand outside of research. Investigators use it for extortion letters or threatening emails. A famous case was the “Unabomber” in the USA: his brother recognized typical phrasings, and linguistic analyses supported the suspicion. Publishers use stylometry when someone publishes under a pseudonym. This is how it was revealed in 2013 that a praised crime novel had been written by J. K. Rowling.

Since language models have existed—that is, programs that themselves produce fluent text—the method has gained a new field of application. Schools and universities want to know whether a paper was written by the student themselves. Stylometric procedures are built into many of these checking tools. Their verdict is not certain, however—more on that later.

What gets counted, and why exactly that

Surprisingly, the striking words are of the least interest. Whether someone writes “photosynthesis” depends on the topic, not the person. What’s telling are inconspicuous function words: “and,” “but,” “the,” “to.” They occur in every text, and no one chooses them consciously. That is precisely why their frequencies reveal the person behind the text.

There are further metrics as well. Average sentence length, the distribution of word lengths, punctuation, the share of rare words in the overall text. A classic method is called Delta and comes from the researcher John Burrows. It compares the frequencies of roughly the hundred most common words and calculates from them a distance between two texts. The smaller the distance, the more similar the style.

Modern systems often leave the selection of features to the computer. A learning model is presented with many texts of known authorship and searches out the telltale patterns itself. This is more accurate, but harder to trace. In the end you see a result, not necessarily the reason for it. For courts, this is a problem, because an expert opinion must be explainable there.

Limits, fallacies, and the debate over AI detectors

Stylometry needs material. With a postcard of three sentences it fails; with a novel it works well. As a rough rule of thumb, a few thousand words per text are considered necessary. It also requires comparison texts: without writing samples from the suspected person, nothing can be attributed. So the method can confirm or rule out, but cannot name anyone out of thin air.

A second catch is manipulability. Anyone who knows their style is being measured can disguise it or run the text through a translation program. Editing also blurs traces. And a result is always a probability, not proof like a fingerprint.

This becomes especially visible with tools meant to detect AI texts. Among other things, they measure how predictable the word choice is, since language models often pick the most obvious word. People with a very factual or well-practiced style are thereby unjustly placed under suspicion. This particularly often affects people writing in a foreign language. Several providers have therefore discontinued their detectors again. Anyone reading stylometric results should treat them as an indication, never as a verdict.