Context Switching

A computer appears to do many things at once. Music is playing, a browser is loading a page, in the background a file is copying itself. In fact, a single processor core can only ever work on one thing at a time. It therefore jumps back and forth between tasks very quickly. This very jump is called context switching: the computer remembers where it left off with the first task, turns to the second one, and later retrieves the saved state again. Because the switches happen thousands of times per second, it looks to us like genuine simultaneity.

The price of jumping back and forth

Every switch costs time in which nothing useful is being computed. The state of the old task has to be written out, that of the new one loaded in. A single switch only takes microseconds. But with millions of switches per second, this adds up to a noticeable loss of computing power.

There is also a less visible effect. Processors keep frequently used data in a small, very fast intermediate memory, the cache. After a switch, the wrong data sits there, namely the data of the previous task. The new task first has to fetch its data from the slower main memory. This follow-up cost often takes more time than the switch itself.

That’s why context switching is, in practice, a balancing act. If a system switches too rarely, a long task blocks all others and the program feels sluggish. If it switches too often, it spends a large part of its time on administration instead of on work. In the extreme case, this is called thrashing: the system is only busy with itself.

What is saved during a switch

The context is everything a task needs to keep going. This includes the intermediate results in the registers, which are tiny memory cells directly inside the processor. It also includes the point in the program where the task was interrupted. This information is written to an area of main memory and read back in when returning.

The whole thing is controlled by the operating system, that is, Windows, macOS, Linux, or Android. A part of it, the scheduler, decides which task gets to go next. In doing so, it takes priorities into account. A mouse movement or a keystroke must not wait, a download in the background can.

A comparison helps: you’re doing math homework and switch to English. You place a bookmark in the book and push the papers aside. When switching back, you first have to get back into it. This very re-immersion is the actual bottleneck, for humans as for processors.

From graphics cards to the productivity debate

In the AI world, the term comes up in connection with data centers. If a graphics card is supposed to serve several models at the same time, it has to switch between them. Because models bring along many gigabytes of numerical values, this switch is expensive. Providers therefore try to avoid it and keep a model running on the same card for as long as possible.

Outside of technology, context switching is used as a buzzword in debates about work. What is meant then is the constant switching between chat, email, and the actual task. Studies on attention show that after an interruption, people need many minutes to get deeply back into something. So switching off notifications doesn’t just save the seconds spent reading them.

A common misconception is to equate context switching with multitasking. Multitasking describes the goal of advancing several tasks in parallel. Context switching is the technique by which a single processor core can even pretend to achieve this goal in the first place. Genuine simultaneity only arises through multiple cores that actually work alongside one another.