
Out-of-Order Execution
Out-of-order execution is a technique in modern processors in which computational instructions are not processed strictly in the order they were programmed, but rather in the order in which the required data becomes available. This avoids unnecessary waiting periods and makes the chip significantly faster.
A processor — the main computing component in every computer or smartphone — normally works through a list of instructions, step by step. Sometimes, however, it has to wait for a result before it can continue, for example because a value first has to be loaded from main memory. That takes time. Instead of remaining idle during this wait, with out-of-order execution the processor simply looks further ahead in the instruction list and executes whatever is already possible. In the end, the results are assembled as if everything had happened in the correct order. The program doesn’t notice any of this — it gets exactly the result it would expect.
Why order slows things down
Modern processors are extremely fast — a single core today manages several billion computing operations per second. Main memory, by contrast, is comparatively slow. An instruction often takes a single clock cycle, but loading a value from memory can take up to 200 cycles. Without out-of-order execution, the processor would simply wait during this time. That would be like a craftsman only fetching his tools when he needs them, instead of laying everything out in advance.
The performance difference is substantial. Measurements show that a processor with out-of-order execution can often process twice as many instructions per clock cycle on typical software as a simple, strictly sequential processor at the same clock frequency. Clock frequency — that is, how often the processor “ticks” per second — therefore says relatively little on its own about actual speed.
How the processor looks ahead
Inside the processor there is a buffer known as the reservation station or instruction window. Incoming instructions first land there before being executed. A control logic continuously checks which instructions already have all the input values they need. Exactly these are executed next — regardless of where they stood in the original program.
To ensure nothing gets mixed up in the process, the processor numbers the instructions internally. A separate component, the reorder buffer, collects the completed results and passes them on externally in their original order. This means the program and the operating system always see an orderly, correct result. The disorder remains an internal matter of the chip.
It’s important to distinguish this from a related technique: pipelining means that several instructions are simultaneously at different stages of processing — one is being loaded, another is being computed, a third is being stored. Out-of-order execution goes further: it additionally changes the order of execution itself. Many processors today use both techniques at once.
Out-of-order execution in products and headlines
Nearly every high-performance processor since the 1990s uses out-of-order execution: Intel's Core series, AMD's Ryzen, Apple's M-series chips, and the processors in current smartphones. It is one of the core techniques that ensure apps run smoothly even though the underlying code was not explicitly written for parallelism.
Out-of-order execution made prominent headlines in 2018. That’s when the security vulnerabilities Spectre and Meltdown became public. Both exploited the fact that, during out-of-order execution, processors sometimes execute instructions that should never have been executed in hindsight — and in doing so, still leave traces in the chip that attackers can read out. The vulnerabilities affected nearly all modern processors worldwide and showed that an optimization technique can also carry unforeseen risks.
The topic is also relevant for AI chips. Specialized accelerators such as Google's TPU or Nvidia's GPU cores, by contrast, are often deliberately built simpler and forgo out-of-order execution in order to save space and energy. They gain their speed not through lookahead, but through sheer parallelism — thousands of simple computing cores running simultaneously.