Beam Width
The beam width determines how many possible sentence beginnings a language program pursues simultaneously before settling on an output. A larger value often delivers better results, but costs more computing time.
Programs that generate or translate text write their answer word by word. At every step there are several plausible continuations. One can always take only the single continuation that looks best at the moment. But one can also write several variants forward in parallel and only pick the best one at the very end. The beam width is the number that specifies how many variants are kept open at the same time in this process. With a beam width of 5, five half-finished sentences always run side by side until one of them wins.
Why a single number decides the quality of translations
The problem with writing word by word is called short-sightedness. A word may fit well right now and still lead the sentence into a dead end. Anyone who only makes the best individual decision at each step notices this only when it is too late. The beam width is the lever used to soften this short-sightedness.
A comparison: in chess, a beginner thinks only about the next move. A good player keeps several move sequences in mind and decides later. A larger beam width makes exactly this kind of difference. In machine translation, values between 4 and 10 were long the standard, because they measurably produced better sentences than the simple approach.
The price is computing time. A beam width of 10 roughly means that ten times as much computation is required as with a single variant. With millions of requests per day, this becomes a real cost factor. That is why choosing this number is always a trade-off between quality and effort.
How the candidates are filtered
The procedure behind this is called beam search. It works in rounds. In each round, the program takes the currently open variants and appends every conceivable next word to each of them. This produces a huge set of new partial sentences. From this set, the program keeps only the best ones — exactly as many as the beam width specifies. All others are discarded.
Evaluation is done via probabilities. The model estimates, for every possible next word, how well it fits. The values from the individual steps are combined into an overall score for the partial sentence. A partial sentence with a weak start can therefore still recover if the following words fit very well. This is precisely the advantage over the short-sighted approach.
A common misconception is that more is always better. Beyond a certain size, results actually get worse again. Very wide searches tend to favor short, boring sentences, because short sequences are often scored better in computational terms. Experts call this the beam search paradox. In practice, this is compensated for with a length correction that mildly penalizes short answers.
Where this dial shows up in products
The technique is most visible in translation services and in speech recognition. When a dictation program retroactively corrects a phrasing in the middle of a sentence, you are watching beam search at work. Systems that complete program code or describe images also make use of the procedure.
With modern chatbots, the situation is different. There, the search is usually not for the most probable sentence; instead, words are drawn randomly from among the plausible candidates. This is called sampling, and it makes the answers more varied. Beam width plays hardly any role there, whereas in classic translation systems it very much does. Anyone who confuses the two approaches easily misunderstands discussions about model settings.
In technical descriptions and research reports, the number is often listed casually in a table, for instance as beam size = 4. This is not a minor detail. When comparing two systems with different beam widths, one is also comparing different amounts of computational effort. Serious comparisons therefore always state the value alongside the results.