Inpainting

A trash can appears in a vacation photo. You mark it with your finger or the mouse and delete it. What remains is a hole that needs to be filled. Inpainting is the term for the automatic filling of this gap: the program generates new image content that matches the surroundings. It doesn’t simply copy neighboring image regions, but invents plausible details—that is, details that appear credible. The term originates from the restoration of paintings, where experts repaint missing areas in old paint.

From Retouching Craft to One-Click Tool

In the past, removing an object from a photo was genuine manual labor. A skilled photo editor needed anywhere from minutes to hours, depending on the subject. They had to manually recreate color gradients, shadow edges, and textures like grass or wood grain. Today, a smartphone accomplishes the same task in a few seconds. This has made a skill that once required specialized knowledge available to everyone.

This has an uncomfortable flip side. When every phone can remove people or objects from images without a trace, a photo loses part of its evidentiary value. For journalism, insurance claims, and courts, this is a serious problem. That’s why camera manufacturers and software companies are working on provenance credentials that record whether and how an image was edited. One well-known standard for this is called C2PA.

Economically, inpainting is one of the reasons why image AI generates revenue quickly. Pure image generators create something entirely new, which is often unusable for businesses. Inpainting, by contrast, improves existing material: product photos, real estate images, advertising visuals. You retain control over the image and only change one spot.

Mask, Noise, and Context

The user provides the model with two things: the image and a mask. The mask is a kind of stencil and marks exactly the area that should be newly generated. Everything outside the mask remains unchanged. Often a text prompt is added as well, such as “empty street” or “meadow with stones.” This allows control over what fills the gap.

Most current systems work with so-called diffusion models. During training, these models learned to progressively work out a clear image from an image full of random pixels—that is, from pure noise. In inpainting, this process starts only within the mask. At every computational step, the model looks at the unchanged surroundings and aligns the new pixels accordingly. This produces colors, brightness, and patterns that match the rest of the image.

Related but not identical is outpainting. This involves extending an image beyond its original edge, for example from a portrait format to a widescreen format. Another common misconception is the assumption that inpainting reconstructs the true content behind the removed object. The model doesn’t know what was actually there. It produces a convincing invention, not a restoration.

From the Phone Eraser to Film Production

Most commonly, you encounter inpainting in your smartphone’s photo gallery. Google calls the feature “Magic Eraser,” while Apple offers something similar under “Clean Up.” In Photoshop, the tool is called “Generative Fill.” In all cases, the operation is the same: mark, wait, check the result.

Professionally, the technique is used in film studios. There, camera cables, drones, or barriers vanish from shots where they accidentally ended up in frame. Online stores remove distracting backgrounds from product images. Museums and archives use related methods to close cracks and scratches in old photographs.

In the news, inpainting usually comes up in two contexts. First, with new generations of phones, where photo editing is a key selling point. Second, in debates about manipulated images on social networks. Anyone who understands the technology can better make sense of both kinds of stories.