Voice Bot

A voice bot is a program you can talk to as if talking to a human. You say a sentence, the program understands it and replies with an artificially generated voice. Most often you encounter such programs on the phone, for instance on the hotline of a bank or a health insurer. The difference from the old telephone menus is huge: in the past you had to press buttons or say rigid keywords, today you may formulate freely. At its core, a voice bot is thus a chat program that has been given ears and a voice. It doesn’t type, it listens and talks.

Why companies hand their hotline over to software

Phone support is expensive. A conversation with a human in a call center costs several euros, depending on the country and the request. A very large share of these calls revolves around the same questions again and again. Where is my package? How do I change my address? When will the technician arrive? A voice bot can handle exactly such standard cases around the clock, even at night and on holidays.

For callers, this initially means shorter waiting times. But it also means a new kind of frustration when the request doesn’t fit the pattern. Many people then try to bypass the bot by repeatedly saying “agent.” Good systems recognize this desire and transfer the call. Bad systems go in circles and annoy exactly those customers who have a real problem.

Economically, the topic is exciting because it concerns many jobs. Call centers employ millions of people worldwide. When software takes over part of these conversations, the work shifts: humans then mainly handle the difficult and emotional cases. That’s why voice bots regularly appear in business news and quarterly reports.

From sound to answer and back

A voice bot consists of three parts connected in sequence. The first part is speech recognition. It turns the sound waves from the microphone into written text. The second part is the actual language model, i.e. a program that has learned from huge amounts of text to formulate sensible answers. The third part is speech synthesis, which converts the response text back into audible speech.

Between step two and the answer there is usually also access to the company’s databases. Without this access, the bot could know nothing about your order. Nor is it allowed to freely make things up; instead it should stick to stored rules. Experts call it hallucination when a language model invents things that sound plausible but are wrong. On a hotline, this would be especially unpleasant, for example with a fabricated notice period.

The biggest technical hurdle is time. A phone conversation only feels natural if the answer comes after about half a second. All three steps must therefore run very quickly. Modern systems already start answering before the sentence has been fully processed. Just as tricky is handling interruptions: the bot must notice when the caller talks over it and then fall silent immediately.

From the kitchen speaker to the fraud scheme

The best-known representatives sit in living rooms and kitchens. Alexa, Siri, and the Google Assistant are voice bots, even though they are usually called voice assistants. They answer short questions, set alarms, and control lamps. Their conversational ability was long limited, but is currently being significantly expanded by modern language models.

In working life, voice bots can be found in medical practices that take appointments, and in car repair shops that manage callbacks. Such a system is also often built into the car itself. In training, some companies use voice bots as practice partners for sales conversations.

There is a dark side. Voices can now be recreated from just a few seconds of recording. Fraudsters use this to call relatives and pose as family members in distress. That’s why there is discussion about whether bots on the phone must disclose that they are not human. The European Union’s AI Act requires exactly such a disclosure for systems that interact directly with people.