Data Loader
A data loader is the piece of software that fetches training data from storage, prepares it in portions, and passes it on to a learning program. It plays a key role in determining whether expensive computing chips are kept busy or left waiting unnecessarily.
For a computer program to learn from examples, it first has to receive those examples. Images, texts, or measurements sit on a hard drive or on a server somewhere on the internet. The data loader is the part of the software that fetches this data and passes it on in a usable form. It doesn't deliver everything at once, but in small portions called batches. A batch typically consists of 32 or 64 examples that are processed together. You can picture the data loader like the serving line in a cafeteria: it makes sure a tray is always ready as soon as the next guest arrives.
Why the supply chain determines training time
The actual computation during learning is handled by graphics chips, known as GPUs. These chips are extremely fast and extremely expensive. A powerful GPU in a data center costs several euros per hour. When it's waiting for data, the meter keeps running regardless.
This is exactly where a frequently underestimated problem lies. Hard drives deliver data far more slowly than a GPU can process it. If the supply arrives too late, the chip sits idle. Experts refer to this as a data-loading bottleneck. In practice, a poorly built data loader can make training take twice as long as necessary.
That's why the data loader is not a side issue, but a field of optimization in its own right. Raising GPU utilization from 40 to 90 percent saves real money. For large training runs, this can amount to sums in the six or seven figures.
From raw image to finished batch
A data loader carries out several steps in sequence. First, it reads in a file, say a photo in JPEG format. Then it converts it into numbers, since a learning program only computes with numbers. Next, it brings all examples to the same size so they can be processed together.
Often an alteration of the data is added, called augmentation. Images are flipped, slightly rotated, or shifted in their crop. This way the model never sees the exact same photo identically twice and learns to be more robust. In addition, the data loader shuffles the order of the examples anew on every pass. Otherwise the model might memorize the order instead of the content.
To make all of this fast, a data loader works in parallel. Several worker processes prepare the next batches simultaneously while the GPU is still computing on the current one. This working-ahead is called prefetching. The finished portions wait in a queue in working memory. In the ideal case, the computing chip never has to pause even for a single second.
Where data loaders show up in practice
Anyone who programs with AI themselves encounters the term immediately. The popular toolkits PyTorch and TensorFlow come with ready-made data loader building blocks. In PyTorch, the corresponding component is simply called DataLoader and is used in nearly every tutorial within the first few lines. You specify how large a batch should be and how many processes should work in parallel.
In news coverage about AI companies, the topic tends to appear more indirectly. Whenever data pipelines, data preparation, or data quality are mentioned, the data loader is involved. With very large language models, text volumes in the range of trillions of words are passed through in this way. Such amounts of data don't fit into any working memory; they have to be continuously reloaded.
It's important to distinguish this from a related term in enterprise IT. There, data loader sometimes refers to a tool that imports datasets into a database or a system like Salesforce. That has nothing to do with machine learning. In the AI context, it always refers to the supply of training data.