Self-hosted Deployment
A self-hosted deployment means that software runs on computers that the user owns themselves or rents and manages themselves – not on those of the manufacturer. With AI systems, it is mainly companies and government agencies that choose this path, because their data never leaves their own network.
When you use a chatbot in your browser, the actual computing work does not happen on your laptop. It happens on large computers belonging to the provider, somewhere in a data center. Your input is sent there, and the answer comes back. With a self-hosted deployment, things are different: the software is installed on machines that one owns or has rented oneself. So instead of getting a finished service, you get the program itself – and take care of running it yourself. The English word “deployment” means roughly “putting into operation” or “rolling out” a piece of software.
When data may not leave the building
The most important reason for this extra effort is the data. A law firm that has an AI summarize legal briefs is thereby sending confidential client files away. A hospital would be transmitting patient data, a bank account transactions. In many of these cases, laws or contracts specifically forbid exactly that. If the model runs in the company’s own server room, not a single byte leaves the building.
On top of that comes independence from the provider. A cloud service can raise prices, discontinue features, or replace a model with a new one. Whoever hosts it themselves decides alone when an update happens. For software that is embedded in a production plant, that can matter more than having the latest version.
The price for this is work. You need people who maintain servers, patch security vulnerabilities, and are reachable at night if something fails. For small teams, this often eats up more money than the cloud service would have cost. Self-hosted is therefore not automatically the cheaper choice, but the more controlled one.
From model download to running server
The foundation is usually open models whose weights may be freely downloaded. Weights are the numbers a model has learned during training – essentially the file in which its knowledge is stored. Well-known examples are Llama from Meta or the models from Mistral. A model from a company that only offers access over the internet cannot, in principle, be self-hosted.
The file alone is not enough. You need graphics cards with enough memory, because large models simply won’t fit otherwise. On top of that comes server software that receives requests, passes them on to the model, and sends back answers. Common tools for this include vLLM or Ollama. The whole setup is often packaged in containers – self-contained packages that bundle the program together with all necessary supporting files.
A common misconception is that self-hosting always means your own basement. A server rented from a provider also counts, as long as you manage it yourself and run the software yourself. What matters is control over the operation, not the location of the hardware.
Where the term shows up in news reports
In business news, self-hosting almost always appears in connection with data protection. When government agencies or corporations announce that they are running AI “on their own infrastructure,” this is exactly what is meant. The word on-premise often comes up too, which essentially describes the same thing.
The idea is old outside of AI as well. Anyone who runs their own Nextcloud instead of Dropbox is self-hosting. Anyone who runs a Minecraft server on an old computer is doing the same. The difference with AI lies mainly in hardware costs: graphics cards for large models are expensive and in short supply.
For providers of open models, self-hosting is a selling point against the big cloud companies. That is why press releases emphasize that a model can be “run locally.” For small models, this even holds true on good laptops.