Regex

If you search a document for the word “appointment,” you type in exactly that word. You’ll then only find that exact sequence of letters. But often you want to find something more general: all phone numbers, all dates, all words with a double “s.” A regex is a short string of characters that describes exactly such a search pattern. Instead of saying “find this word,” you say “find everything that’s structured like this.” The full name is regular expression, though almost nobody uses it in everyday speech.

Why programmers would be lost without pattern search

A huge amount of work with computers consists of searching through and cleaning up text. A company has a list of 50,000 customer records and wants to find all invalid email addresses. Doing this by hand is impossible, and a normal search won’t cut it either. With a regex, it’s one line of code.

The second reason is consistency. Regex works almost the same way everywhere: in Python, in JavaScript, in the search fields of editors like VS Code, in server programs. Once you understand the notation, you can use it for decades across all kinds of different tools. The basic idea dates back to the 1950s and has barely changed since.

Regex does have a bad reputation, though, and it’s partly deserved. The patterns are written extremely tersely and are therefore hard to read. An expression you wrote yourself three months ago often looks like a foreign language. That’s why the rule of thumb applies: regex is powerful for simple patterns and a bad idea for complicated structures like entire HTML pages.

The building blocks of a pattern

A regex consists of ordinary characters and special characters with special meaning. A period stands for “any character at all.” An asterisk after a character means “any number of times, including zero.” In square brackets you list allowed characters: [aeiou] matches any vowel.

One example: the pattern \d searches for exactly four digits in a row. This lets you find years like 2024 or zip codes. The \d is shorthand for “a digit,” and the curly braces specify the count. Longer patterns are assembled from such small building blocks.

What matters is how the program processes the pattern. It goes through the text character by character and tries whether the pattern matches starting at that position. If it doesn’t match, it moves one position ahead. With poorly constructed patterns, this trial-and-error process can explode, because the program ends up testing millions of combinations. Such cases are called catastrophic backtracking, and they have already brought major websites down for hours.

Regex in everyday life and in AI tools

You encounter regex most of the time without even noticing. When a signup form complains that your email address is invalid, there’s often a regex checking it behind the scenes. Password rules like “at least one number and one special character” are also frequently enforced this way. In the search fields of programming editors, there’s almost always a small toggle that switches on pattern search.

In the context of artificial intelligence, regex shows up in a surprising place. Before a language model is trained on text, huge amounts of data have to be cleaned up. Ad lines, control characters, and broken webpage remnants are often filtered out using regex patterns. Chatbot responses are also frequently searched this way, for instance to cleanly extract a number or a block of program code.

And there’s a new division of labor emerging. Because regex is hard to write but easy to explain, many developers now have a chatbot generate the patterns for them. This often works well, but it doesn’t replace testing. A pattern that matches in nine cases and silently fails in the tenth otherwise only gets noticed once the data is already wrong.