Jagged Technological Frontier

Programs that write texts or answer questions are astonishingly strong at some tasks and surprisingly weak at others. The strange thing about this: for humans, it’s nearly impossible to guess which task falls into which group. Such a program can meaningfully summarize a legal contract. Yet it may fail at the question of how many letters the word “strawberry” has. Researchers at Harvard Business School coined the image of a jagged boundary for this in 2023: Jagged Technological Frontier. The capabilities don’t end at a straight line, but jump like a mountain range with peaks and deep valleys.

Why users trust AI both too much and too little

Humans constantly infer one performance from another. Someone who passes a difficult math exam can probably also handle simple percentage calculations. For AI systems, this inference does not hold. That’s precisely why the jagged boundary is more than a curiosity: it destroys our normal judgment about competence.

This leads to two opposite mistakes. Some trust the system too much because it was brilliant in one instance, and uncritically adopt a wrong number or a fabricated quote. Others lose all trust after a silly mistake and stop using the tool even where it would be clearly superior. Both cost money and time.

The Harvard study mentioned, conducted with consultants from Boston Consulting Group, showed this clearly. For tasks within the model’s capabilities, the consultants worked faster and better with AI. For a task deliberately placed outside its capabilities, however, the AI users were wrong more often than the group without AI. The tool did not help neutrally—it actively caused harm at the wrong point.

Where the jaggedness comes from

A language model has not learned to understand the world. It has learned from vast amounts of text which word fragments typically follow one another. What appeared frequently and in many variations in the training data, it masters well. What appeared rarely, or does not exist in text form at all, remains a gap.

On top of this comes the architecture. Models do not break text down into letters, but into larger chunks, so-called tokens. A word is often a single block for the model, without visible individual letters. This is why letter puzzles are hard, while sophisticated summaries are easy. For a human, the difficulty is sorted exactly the other way around.

The boundary also keeps shifting continuously. Every new model generation fills in some valleys and sometimes tears open new ones. A weakness everyone laughed about a year ago may be fixed today. A fixed list of things AI cannot do therefore becomes outdated quickly.

Finding the jagged edges in your own daily work

In everyday life, one encounters this effect with every chatbot. It formulates a job application convincingly and then miscalculates a simple installment payment. It knows the structure of a poem but invents a source for it. Anyone who dismisses this as random sloppiness has failed to recognize the pattern.

In companies, this has become a task of its own. Teams systematically test which of their work steps lie within and which lie outside the boundary. That’s why the term often comes up in business news when it’s about productivity through AI. Studies with contradictory results are frequently explained by the fact that they tested different points of the jagged boundary.

A related but narrower term is hallucination, meaning a fabricated false statement. It is just one manifestation of the problem. The Jagged Technological Frontier refers to the bigger pattern: performance and perceived difficulty are not correlated with AI. The practical consequence is unspectacular but effective. One checks results particularly carefully where one can judge the answer oneself.