Merge

On a larger software project, it’s rare for everyone to work on the same file. Instead, each person creates their own copy of the project and changes whatever they want to change in it. At some point, these copies need to flow back together into a single version. This process of bringing them together is called a merge. A program compares the different versions line by line and builds a common one from them. Today the term is also used outside of programming, for instance when two fully trained AI systems are fused into one.

Why teams work separately in the first place

Without a merge, every change would have to apply to everyone immediately. A half-finished feature would then block the work of all colleagues. That’s why people work in development strands called branches. Each branch is its own construction site. Only once the work there is finished and tested does it move back into the main version via a merge.

This approach is the reason why thousands of people can write on the same project simultaneously. Large open-source projects like the Linux kernel receive several thousand contributions from hundreds of participants per version. Without a clean procedure for merging, this would simply be impossible. The merge is therefore less a technical detail than the organizational foundation of modern software development.

Merging also has a practical reason with AI models: retraining a model from scratch often costs millions. Fusing two existing models, by contrast, costs only a few hours of computing time.

What actually happens during merging

With software, the tool compares three versions: the common starting point and the two changed versions. If only one side changes a line, that change is simply adopted. If the changes affect different parts of the file, this happens automatically and without any prompt. Humans usually don’t even notice.

It gets difficult when both sides have changed the same line differently. That creates a merge conflict. The program refuses to guess and marks the spot in the text. A human has to decide which version applies or whether a third variant is needed. Conflicts are normal and not a sign of an error.

Merging AI models works differently. A model consists of many billions of numbers, so-called weights, which arose during training. A merge simply forms averages from these, often with different weightings of the two sources. This only works if both models have the same architecture and the same origin. If they are related, the result can show abilities of both parents, for instance programming and good German.

From pull request to model zoo

Anyone active on platforms like GitHub or GitLab constantly encounters the term. There, changes are proposed with a pull request; on GitLab it’s called a merge request. Others read the proposal, comment on it, and eventually press the merge button. For companies, this moment is a quality check: nothing lands in the main version that no one has reviewed.

In AI news, merge comes up mainly in connection with freely available language models. The platform Hugging Face hosts tens of thousands of models, many of which are pure merges of other models. Small teams can thus create competitive systems without a large data center.

A common misconception is that a merge adds knowledge together like two buckets of water poured into one. That’s not accurate. Averaged weights can also weaken the abilities of both models instead of uniting them. Whether a merge has succeeded only becomes clear through testing. And beware of the terminological overlap: the Merge in connection with the cryptocurrency Ethereum means something completely different, namely the network’s switch to a more energy-efficient method in 2022.