Modal Labs

Anyone working with artificial intelligence today needs a lot of computing power. Your own laptop isn’t enough for that, so you rent machines in large data centers. That’s exactly what the company Modal Labs, based in New York, sells: access to other people’s computers, which you only pay for during the minutes they’re actually computing. The developer writes their program as usual on their own device and simply marks in the code which parts should run on a powerful machine. Modal takes care of everything else: finding a machine, starting it, sending the program to it, returning the result. The company was founded in 2021 and is financed by venture capitalists, meaning investors who bet on the rapid growth of young companies.

Why developers pay for this

The machines that train or run AI models are expensive. They’re usually graphics cards originally invented for video games, but which happen to be especially good at performing many calculations simultaneously. A single such card can cost several tens of thousands of euros. Hardly any small team buys them outright if they only need them for a few hours at a time.

The real advantage of Modal, however, is the saved work time. Setting up servers is its own field of expertise full of pitfalls: operating systems, drivers, network settings, security rules. A two-person startup has no one for that. So Modal doesn’t just sell electricity and silicon, but above all the absence of configuration work.

The billing model is also important. Traditionally, you rent a server per month and pay even when it’s doing nothing. With Modal, the machine only runs during the request and is switched off immediately afterward. For applications with irregular usage, such as an image AI with only a few users a day, this is considerably cheaper.

The path from code to graphics card

Technically, Modal belongs to the serverless category: the server obviously exists, but the developer never sees it and doesn’t manage it. They write a normal function in the Python programming language and place a short line above it saying: this should run in the cloud, with this graphics card and these additional programs. On startup, Modal sends the code and the required environment to the data center.

The core problem here is the cold start. When a request comes in and no machine is ready, one first has to be booted up. With typical systems, this takes minutes because an AI model has to load many gigabytes of learned values from disk. Modal has built its own file system for this that provides this data faster. Startup times of just a few seconds are advertised.

If many requests suddenly arrive, the system automatically starts additional machines in parallel. This property is called scaling. When the load drops again, the machines disappear. For the developer, the code stays the same, regardless of whether one user or a thousand are accessing it.

Modal in the market and in the headlines

You rarely encounter Modal Labs as an end user; it’s usually hidden behind other products. If an app generates an image from a text prompt or tags a video, the computation behind it might be running on Modal. The service is also used in research and for fine-tuning, meaning the retraining of finished models for a specific task.

In business news, the company mainly appears because of its funding rounds. It’s seen as an example that it’s not only the providers of the models who profit from the AI boom, but also the companies in the infrastructure layer beneath them. Competitors include specialized providers such as Replicate or Baseten, as well as the major cloud corporations Amazon, Microsoft, and Google.

A common misconception: Modal Labs does not build its own AI models and is not a competitor of OpenAI. The company sells the execution environment, not the intelligence. A comparison helps: OpenAI would be the car manufacturer, Modal the workshop with a lift that you rent by the hour.