Micro-VM

A server in a data center doesn’t serve just one customer, but hundreds. So that these customers' programs cannot see or interfere with each other, the server is split into several isolated computers using software. Each of these simulated computers has its own operating system and believes it is alone on the machine. Such simulated computers are called virtual machines. A micro-VM is a particularly lean variant of this: it gets only the absolute minimum of simulated components, such as a network card and a hard drive, but no graphics output, no sound, and no USB ports. That’s why it starts in about a hundred milliseconds instead of half a minute, and sometimes occupies only a few megabytes of memory.

Why cloud providers switched to it

Classic virtual machines are secure, but sluggish and memory-hungry. They are suited for servers that run continuously for months. Modern cloud services work differently: a tiny program is started because someone loads a webpage, and after 200 milliseconds it’s gone again. For this rhythm, a heavy virtual machine would be completely unusable.

The second reason is security. There is a lighter alternative to virtualization, namely containers. Containers share the kernel of the operating system, that is, the innermost control part of the host machine. If an attacker finds a vulnerability there, they may be able to break out of their container and reach the neighbors. A micro-VM, by contrast, has its own operating system kernel and a real partition wall supported by the hardware.

Micro-VMs thus combine both: the hard isolation of a virtual machine with a startup time that comes close to that of a container. This exact combination was long the bottleneck in operating cloud platforms that run foreign code. This is economically relevant because it allows a server to support more paying customers at the same time.

What gets left out when slimming down

A normal virtual machine behaves as if it were a complete PC. It simulates a BIOS, i.e. the startup software of a real computer, plus drives, ports, and screen output. This simulation costs time and program code. A micro-VM leaves almost all of this out and boots directly into the operating system kernel.

The advantage is twofold. First, startup is faster because fewer components need to be initialized. Second, the risk decreases: every simulated component is program code, and any program code can contain bugs through which an attacker could break out. Fewer components thus mean a smaller attack surface. Experts refer to this as a small attack surface.

The best-known tool for this is called Firecracker and was developed at Amazon. It is deliberately built to know only a handful of simulated devices. A common misconception is that a micro-VM is simply an especially small container. This is not true: the isolation works via a different mechanism, namely real virtualization with support from the processor.

Micro-VMs in cloud services and AI products

Anyone running a function on AWS Lambda is, in all likelihood, working inside a micro-VM without noticing it. The same applies to many services where you type code into a browser window and have it executed immediately. Online programming courses and automated testing systems also use this technology.

It has become especially important because of AI assistants that write and execute programs themselves. This code is unverified and can be flawed or even malicious. It is therefore run inside a micro-VM that is simply discarded after the task is done. In technical writing, such an isolated test area is called a sandbox.

In business news, you’ll usually encounter the term in connection with costs and utilization. When a provider reports being able to serve more customers per server or shorten cold starts, this technology is often behind it. A cold start refers to the waiting time when an environment has to be freshly booted up.