Endpoint

Programs need to be able to talk to each other. For that to work, every program needs a fixed address to which a request can be sent. That’s exactly what an endpoint is: a clearly named location on the network where a service accepts requests and sends back responses. It usually looks like an internet address, such as api.example.com/v1/chat. The difference to a normal website is the recipient. A website is made for humans, an endpoint for other programs. That’s why what comes back isn’t a colorful page, but plain text containing the requested data.

The handover point between foreign code and the model

Almost every app that uses AI doesn’t have the model built in itself. Large language models need expensive graphics cards and a lot of electricity. So they run in data centers, and the app sends its requests there. The endpoint is the only point at which the app comes into contact with the model. Everything else remains invisible to it.

This has a practical advantage. The provider can swap out, improve, or move the model to other servers behind the endpoint. As long as the address and the shape of the request stay the same, the app notices nothing of this. Conversely: if an endpoint is shut down, every program that used it stops working. This happens regularly when providers discontinue older model versions.

For companies, the endpoint is also the place where billing and monitoring happen. Every request is counted there. That’s why endpoints constantly show up in contracts and price lists of cloud providers.

Request, key, and response

A request to an endpoint consists of three parts. First, the address itself. Second, a key, the so-called API key, which acts like a password proving that one is allowed to use the service. Third, the actual data, for example the question to the model and the name of the desired model.

This data is sent in a fixed format, usually JSON. That’s a simple notation for text blocks with labels that programs can easily read. The server checks the key, passes the question on to the model, and sends the answer back in the same format. This is often accompanied by additional information, such as how many words were processed.

You can think of an endpoint like an order window at a food stand. You know where the window is, and you know how to order. What happens in the kitchen behind it doesn’t matter to you. What matters is only that the order arrives in the expected form. A typo in the format doesn’t lead to a bad answer, but to an error message.

Endpoints in apps, news, and IT security

Anyone who uses a chat app, a translator, or image generation in the browser is constantly talking to endpoints without noticing it. School platforms, banking apps, and weather widgets also fetch their data this way. When programming, you encounter the term at the latest when you want to build an AI model into your own project. Providers offer a list of their endpoints for this, along with the permitted request formats.

In business news, you often read that a company has made a new model “available via the API.” What’s meant is that there is an endpoint for it and other companies can program against it. That’s often the actual business purpose: it’s not the company’s own app that brings in the money, but the millions of requests from outside developers.

Beware of a mix-up. In IT security, “endpoint” means something different, namely a device at the edge of the network, i.e. laptop, phone, or printer. Products there are called “Endpoint Protection.” Which meaning is intended can be recognized from the context: if it’s about interfaces and requests, the network address is meant; if it’s about virus protection, devices are meant.