Turn-Taking

A conversation is neither a monologue nor chaos — it follows rules. Humans intuitively know when it’s their turn to speak, when to pause, and how to hand over the floor. This orderly alternation between conversation partners is called turn-taking. A “turn” here is a single conversational move: everything one person says before the other takes over. In linguistics, turn-taking has been its own field of research for decades. In AI development, it has become a concrete technical problem: How does a system recognize that a human has finished speaking — and conversely, when is it the system’s turn?

Turn-taking as a key problem for voice assistants

For humans, turn-taking largely happens unconsciously. One listens to sentence melody, pays attention to brief pauses, and reads facial expressions and gestures. A falling voice at the end of a sentence often signals: I’m done. A brief “hm” or eye contact says: I’m still listening, keep talking. The brain processes these signals in real time, without conscious thought.

For an AI system, none of this is a given. A voice assistant that only waits for silence interrupts mid-sentence — or waits too long and seems sluggish. Errors in turn-taking disrupt a conversation massively. Anyone who has ever talked to a poorly tuned phone bot knows the frustrating feeling of constantly talking past each other. This shows: how well an AI listens and responds is at least as important as what it says in terms of content.

How AI systems recognize the right moment

Modern speech systems combine several methods to solve turn-taking. One of them is endpoint detection (“end-of-turn detection”): the system analyzes volume, speech pauses, and speech melody — that is, whether the voice falls or rises at the end of a sentence. A rising tone suggests a question, a falling one suggests an ending. This sounds simple but is error-prone, because people pause mid-sentence, hesitate, or restart their thoughts.

Newer systems use so-called language models (programs that statistically understand text and speech) for this, which take into account not just the sound but also the content of what is said. Is a sentence grammatically complete? Did the person ask a question? Such signals help to better assess the right moment to take over. Some systems also respond with short acknowledgment sounds like “mhm” or “I see” to signal: I’m listening, it’s not my turn yet. This is called backchannel signaling — a technique that humans use quite naturally.

A common misconception is confusing turn-taking with mere transcription — that is, converting speech into text. That is a preliminary stage. Turn-taking goes further: it’s about controlling the flow of conversation itself, not just recognizing words.

Turn-taking in products and current developments

Turn-taking is embedded in almost every product that works with speech. Voice assistants like Siri, Alexa, or Google Assistant must constantly decide whether the user has finished speaking. Call center bots that automatically answer customer inquiries are another example. Translation systems that interpret live in conversations also need functioning turn-taking — otherwise they translate a sentence while the speaker has already moved on to the next one.

This problem became especially apparent in 2024, when several AI companies worked on so-called “voice modes” for their chatbots — that is, voice modes intended to enable real conversations in real time. OpenAI introduced such a mode for ChatGPT and explicitly highlighted that the model now reacted more naturally to interruptions and could itself better recognize when it was its turn. Precisely this advance was described as one of the most difficult technical steps — even though it seems completely self-evident to any human in any conversation.

Turn-taking is thus a good example of how the hardest problems in AI are often not mathematical puzzles, but things humans master without thinking.