Screen Scraping

When you read an account balance on a website and type it into a spreadsheet, you’re doing manual work. Screen scraping is the automated version of this. A program calls up the page, searches for the desired number at the spot where it’s displayed, and saves it. It reads the finished display, just as a human would see it on screen. This is a detour: the data is stored neatly in the background, but the provider doesn’t release it directly. That’s why the program accesses it where the data is visible anyway.

The detour when there’s no official interface

Normally, programs exchange data through an interface that the provider makes available for that purpose. This is called an API. It delivers numbers in a fixed, machine-readable format, roughly like a neatly filled-out form. Where there’s no API, the screen remains the only data source. Screen scraping is therefore rarely the most elegant solution, but often the only one available.

Economically, the technique is surprisingly significant. Many finance apps only came into existence through screen scraping: they logged into users' banks with their credentials and read the transactions off the online banking page. Genuine interfaces only came later. In Europe, the Payment Services Directive PSD2 forced banks from 2018 onward to offer such interfaces. Similar things happen with price comparisons, real estate portals, and market research.

At the same time, the method is controversial. Operators often see it as unwanted use of their content or a burden on their servers. With bank scraping, there’s the additional issue that users have to hand over their password to a third party. That is now considered a security risk and is one of the reasons for the shift to APIs.

From source code to dataset

A website technically consists of HTML, a text file with instructions on what should be displayed where. A scraper downloads this file and searches it for patterns. It knows, for example, that the price is always in a section marked as 'price'. This rule is defined in advance by a human. After that, the program can process thousands of pages in minutes.

It gets trickier with pages whose content is only assembled in the browser. In that case, the scraper starts a complete browser in the background, clicks through it, and reads the result. With old company programs that have no website, people go even further: the screen content is captured as an image and evaluated with text recognition. This is precisely where the name comes from.

The major weakness is fragility. If the operator changes the design, the pattern no longer matches and the scraper delivers nonsense or nothing at all. A common misconception is equating screen scraping with hacking. Nothing is being broken into; only publicly visible content is being read automatically. Whether this is permitted still depends on copyright, data protection, and the terms of use.

Price comparisons, banking apps, and AI training data

The technique is most visible on comparison portals for flights, hotels, or electricity prices. Part of their offerings comes from official partnerships, part is read off provider pages. Software robots in offices also use the same principle: they fetch numbers from an old program and enter them into a new one because there’s no connection between the two. This automation of clicking work goes by the abbreviation RPA.

In the news, screen scraping currently appears mainly in connection with AI. Large language models were trained on texts that were collected en masse from the internet. Publishers and platforms like Reddit or the New York Times are fighting back legally or demanding licensing fees. It’s also new that AI assistants themselves are supposed to operate websites — that is, click, read, and fill in forms. This suddenly turns an old technique into a current point of contention once again.