Automatic Speech Recognition

Automatic Speech Recognition is the technical term for automatic speech recognition. It refers to software that records spoken language and turns it into written text. The computer receives an audio recording and returns letters, words, and sentences. This is often also called Speech-to-Text, literally: speech to text. An important distinction is that the program does not understand the content, it merely writes down what was said. What happens to the text afterward, such as answering a question, is handled by a different program.

Why speech is the most convenient way into the computer

Speaking is significantly faster than typing. A person speaks roughly 120 to 150 words per minute, but rarely types more than 40. This is crucial anywhere the hands are occupied. A doctor can dictate during an examination, a tradesperson can speak a note on a construction site. Without speech recognition, someone would have to type it up afterward.

There is also the aspect of accessibility. Deaf and hard-of-hearing people can only follow videos with subtitles. Automatic subtitles are now generated in seconds instead of hours of manual work. People who have difficulty using a keyboard also control devices with their voice.

Economically, ASR is the entry point to many AI products. Every voice assistant and every meeting-transcription program begins with this step. If the recognition is poor, even the smartest model behind it is of no use. Errors propagate: a misheard name becomes a wrong answer.

From sound-wave image to finished sentence

First, the sound is converted into numbers. The microphone measures air pressure thousands of times per second. From this, the program calculates a kind of image of the recording, a so-called spectrogram. It shows which pitches were how loud at which point in time. For the computer, the task is thus similar to image recognition.

This image then passes through a neural network, that is, a program that has learned from a great many examples. It is trained with thousands of hours of recordings for which the correct text is already available. In the process, the network itself learns which sound patterns correspond to which sounds. In the past, developers set such rules by hand; today this happens automatically.

At the end comes the decision as to which sequence of words is most likely. This is because many words sound alike, for example “there” and “their”. The model uses the context of the sentence to choose. This is precisely where the typical weakness lies: with strong dialects, background noise, or specialized vocabulary, the hit rate drops noticeably. This is measured using the word error rate, the proportion of incorrect words in the result.

ASR in phones, TVs, and meetings

The best-known example is the dictation function on smartphones, the small microphone icon next to the keyboard. Voice assistants such as Siri or Alexa also begin every request with speech recognition. Video platforms generate subtitles automatically, often in multiple languages. Streaming services and TV broadcasters use the same technology for live subtitles.

In professional life, ASR tools appear mainly in video conferencing. Programs transcribe the conversation and then summarize it. Call centers automatically analyze customer conversations. In clinics, dictation software replaces part of the writing work.

In the news, the term usually comes up in connection with models like OpenAI’s Whisper, which is freely available. Data protection is then often discussed: does the recognition run on the user’s own device or on external servers? Fairness is also a topic, since many systems recognize standard language better than accents. Anyone reading such reports should pay attention to the stated word error rate, since only that makes figures comparable.