Expand and Contract

Large online services run around the clock. Nevertheless, their program code must constantly be changed, for instance when a data field gets a new name. Anyone who switches both at the same time risks the service failing for all users. Expand and Contract is an approach that avoids exactly this. You build the new thing first in addition to the old, let both exist side by side for a while, and remove the old one only at the very end. The name describes this movement: first expand, then contract again.

Rebuilding during live operation

A service like an online shop or a chatbot consists of many program parts that talk to each other. These parts are never all updated in the same second. During an update, part of the servers already run the new version while another part still runs the old one. If both versions have different assumptions about the data, errors occur. Expand and Contract ensures that both versions work at every point in time.

The benefit is primarily economic. In the past, such rebuilds were planned for a maintenance window, often at night. The service was then unreachable for hours. But with a worldwide offering, there is no nighttime when nobody is online. Every minute of downtime costs revenue and trust.

There is also a safety aspect. Because the old variant remains functional the whole time, you can switch back at any moment. If something goes wrong during the changeover, you lose hours instead of days. Without this pattern, a way back is often no longer possible once the data has been rewritten.

The three phases of the rebuild

In the expand phase, only additions are made, nothing is ever removed. If, for example, a database column called “phone” is to be renamed “phone_number” in the future, the new column is simply added alongside it. The old column remains untouched. The program then writes to both columns simultaneously for a while and reads from the old one as needed. In this phase nothing can break, because all existing program parts continue to find their familiar column.

Next comes the migration phase. Existing legacy data is copied into the new column, usually in small batches in the background. Reading is then gradually switched over to the new column. The transition is only complete once all program parts have been verifiably switched to using only the new variant.

The contract phase cleans up: the old column is deleted, as is the code that served both variants at the same time. This step is the least popular, because it produces nothing visible. That is exactly why it is often forgotten. The result is systems full of half-finished rebuilds, where nobody knows anymore which column is the correct one. A rebuild following this pattern is only considered complete once the contract phase has also been carried out.

From database field to API version

The pattern is most commonly encountered with databases. Almost every renaming or restructuring in a production system follows this scheme. It is similar with interfaces, so-called APIs, through which external programs access a service. You may add a new field, but you must not simply remove an existing one, because otherwise other companies' programs would crash.

This also plays a role with AI products. When a provider replaces a language model with a new version, the old version usually remains accessible in parallel for months. Developers get a deadline and an announcement that the old version will be shut down. This announcement is nothing other than the announced contract phase.

A related term is feature flag: a switch in the code that lets you turn a new function on and off for individual user groups. Feature flags control who sees the new thing. Expand and Contract, on the other hand, governs how data and interfaces survive the transition. In practice, both are often used together.