Acceptance Rate
The acceptance rate measures what proportion of a computer program's suggestions are actually adopted by users. It is especially, for coding assistants like GitHub Copilot, the most important metric for whether a system is genuinely useful in everyday work.
Many programs today make suggestions before you’ve even typed anything yourself. The phone suggests the next word, a writing program completes an entire sentence. The acceptance rate is the number that indicates how often such suggestions are accepted. To calculate it, you divide the number of adopted suggestions by the number of all suggestions shown. If a hundred suggestions come up and the user adopts thirty of them, the acceptance rate is 30 percent. It is therefore not a measure of intelligence, but a measure of practical usefulness.
What 30 percent reveals about a product
Whether a piece of software is good is hard to measure. You can pit models against each other in tests, but tests are artificial. The acceptance rate, by contrast, measures the real behavior of real people. Nobody voluntarily adopts a suggestion that doesn’t help them. That’s why many companies consider this metric more honest than any benchmark, meaning any standardized comparison test.
For companies, money is also at stake. Every suggestion generated costs computing power, whether it is accepted or not. A low acceptance rate therefore means you are paying for a lot of waste. As the rate rises, the cost per actually used suggestion falls. That’s why providers like GitHub or Google regularly report this figure to investors.
However, one should not overestimate this figure. A high acceptance rate can also mean that the suggestions are very short and trivial. Someone who only completes individual brackets will almost always be accepted. Someone who suggests entire functions will be rejected more often, but saves much more time when successful. The metric alone, therefore, says little without knowing how large the suggestions are.
From keystroke to percentage
Technically, the program logs two events. First: a suggestion was displayed. Second: the user adopted it, usually by pressing the Tab key. These two counters yield the fraction. This data flows back to the provider anonymized, often millions of times a day.
Measurement becomes tricky in edge cases. What counts if someone accepts a suggestion and then immediately deletes half of it again? That’s why some providers additionally measure how much of the adopted text is still in the document after ten minutes. This stricter variant is often called retention, meaning the retention rate. It regularly comes in noticeably lower than the plain acceptance rate.
The acceptance rate should not be confused with a model’s accuracy. Accuracy asks whether an answer is factually correct. The acceptance rate only asks whether a human found it useful. A correct but cumbersome suggestion gets rejected. A terse, half-fitting suggestion is sometimes adopted anyway and then adjusted by hand.
Where the figure shows up in the news
The metric is best known in connection with coding assistants. Over the years, GitHub has cited figures for Copilot in the range of roughly 25 to 35 percent. Such numbers are readily cited when discussing how much AI is truly changing software development. Google has also reported that a growing share of its new code stems from adopted suggestions.
Outside of programming, the term appears in search suggestions, automatic email replies, and translation tools. In the translation industry, it measures how many machine-generated sentences a human leaves unchanged. Note: in other contexts, acceptance rate means something quite different, for instance the proportion of accepted applications at a university. In a tech context, it almost always refers to the suggestion adoption rate.