Database migration

Almost every company stores its information in a database. That is a program that stores large amounts of data in an organized way and retrieves it quickly: customer names, orders, account balances. A database migration is the planned move of this data into a different system. Sometimes only the internal organization of the existing database changes, for instance when a new column for phone numbers is added. The comparison to a move applies broadly: you don’t just pack boxes, you also have to check whether everything still fits at the new location. And the shop should, ideally, remain open the whole time.

Why banks are afraid of it

During a migration, something can be lost that cannot be recreated. A crashed server is annoying, but replaceable. Vanished or corrupted account data is not. That’s why migrations often receive more reviews and approvals than any other IT project.

On top of that comes time pressure. Many systems must run around the clock. An online shop that is shut down for twelve hours loses revenue and trust. Large migrations are therefore often carried out at night or on holidays. Some banks schedule an entire weekend for it and announce it months in advance.

Migrations tend to show up in the business press almost only when they’ve gone wrong. In 2018, the British bank TSB moved the data of around five million customers to a new platform. Afterwards, customers saw other people’s accounts or couldn’t access their money for days. The cleanup cost the institution several hundred million pounds and cost the CEO his job.

From the old shelf to the new one

It starts with taking stock. You describe precisely how the data is stored today. This description is called a schema: it defines which tables exist and which fields they contain. Then you define the target schema. Between the two lies a translation rule, the so-called mapping.

Rarely does everything match up one to one. In the old system, the name might sit in a single field, while in the new one it’s split into first and last name. Such transformations are handled by a program that reads the data, restructures it, and writes it anew. Experts call this three-step process ETL, short for Extract, Transform, Load.

No serious team migrates real data on the first attempt. Several trial runs on copies are standard practice. Afterwards, everything is compared: Are there the same number of records? Do the totals match? Only once this works cleanly multiple times does the real date arrive. In case of emergency, there is a way back, the rollback, which switches back to the previous state.

A gentler variant is parallel operation. Both systems run simultaneously for a while and are constantly reconciled. Only once the new system has proven that it works reliably is the old one switched off. This costs more, but significantly lowers the risk.

When the app suddenly looks different

As a user, you usually notice migrations only indirectly. Maintenance windows on the weekend, a banking app that looks different overnight, or a notice that a service is briefly unavailable: behind this is often a data move. Even switching mobile carriers while keeping your phone number is, at its core, a migration.

In the tech industry, the most common trigger is the move to the cloud. Data migrates from a company’s own servers in the basement to providers like Amazon, Microsoft, or Google. A second major driver is company acquisitions, since two merged companies have two separate customer databases.

A common misconception is that a migration is simply copying. In fact, the move itself is often the smallest part. Most of the effort goes into the cleanup beforehand: duplicate entries, outdated addresses, fields that someone repurposed for something else ten years ago. Anyone who carries over old disorder without checking it will have it right back in the new system.