FIFO Queue

A FIFO queue is a waiting line that works like a supermarket checkout. Whoever gets in line first is served first. The abbreviation FIFO stands for “First In, First Out.” Computers often store things in such queues: print jobs, messages, computing tasks. New entries are appended at the back, and removal always happens at the front. This guarantees that the order is preserved, and no one is allowed to cut in line.

Why order and fairness matter

Many things happen at once in a program, but computing power is limited. A queue therefore decouples the creation of a task from its processing. Whoever sends a message doesn’t have to wait until it’s fully processed. It lands in the queue, and another part of the program picks it up later. This makes systems more resilient, because short bursts of load are simply buffered in the queue.

The FIFO principle also ensures fairness. Without a fixed rule, individual tasks could be stuck waiting forever because new ones keep arriving. Experts call this starvation. With FIFO, this can’t happen: every entry is guaranteed to move forward over time. Correctness, too, often depends on order. If a bank processes a deposit before a withdrawal, the result is different than if it happens the other way around.

Append at the back, remove from the front

At its core, a FIFO queue only knows two commands. “Enqueue” adds an element at the back, “dequeue” removes the frontmost one. Nothing in between may be touched. So, unlike with a list, you can’t access an element in the middle. This restriction is intentional: it keeps the data structure simple, fast, and easy to verify.

Technically, the program keeps track of two positions: the head and the tail of the queue. Because of this, both operations take a constant amount of time, regardless of whether ten or ten million entries are waiting. Many systems use a ring buffer for this purpose—a fixed area of memory in which the head and tail move around in a circle. When the buffer is full, the system has to decide: make things wait, reject them, or discard entries.

The counterpart concept of a stack helps draw the distinction. A stack works on the LIFO principle, “Last In, First Out,” like a stack of plates: the plate placed on top last is taken first. There are also priority queues, in which important entries are allowed to jump ahead. A common misconception is that FIFO also guarantees the order of results when multiple workers process entries in parallel. Removal from the queue is ordered, but who finishes processing first is not fixed by that.

From the print queue to the AI request

You encounter this principle in everyday life when printing. If you send three documents one after another, they come out of the device in exactly that order. Keyboard input, network packets, and video streams are also buffered in queues. The loading bar on a video is nothing other than a filled buffer queue.

In the software world, queues are products in their own right. Systems like Apache Kafka, RabbitMQ, or Amazon SQS move billions of messages between programs every day. Amazon even explicitly offers “FIFO queues,” which cost more than the standard version. The surcharge exists because strict ordering across many servers is technically demanding.

AI services, too, have a queue running in the background. Requests to a language model first land in a queue before graphics cards process them. When a provider talks in the news about long wait times or overloaded servers, this is usually the queue being referred to. Pure FIFO processing is rare there, though: paying customers are often given priority, which, strictly speaking, already makes it a priority queue.