Social Manipulation

People change their minds because they get convinced. That’s normal and happens every day. Social manipulation means something different: someone deliberately influences what many people believe or do, while concealing that they are behind it. Typical means are fabricated news, fake profiles on social networks, or images that never actually existed. The difference from honest advertising lies in the deception about intent and about the sender. In debates about artificial intelligence, the term comes up because programs can now generate such content automatically and in huge quantities.

Why forgeries are suddenly possible on a massive scale

In the past, manipulation was mainly a question of effort. Anyone who wanted to write a thousand convincing comments needed an office full of people. Such offices did in fact exist, for example the notorious Russian Internet Research Agency. Today, a language model does this work in minutes, for just a few euros in computing costs.

This shifts the balance of power. Individuals or small groups can run campaigns that were previously reserved for states. At the same time, the quality improves: texts no longer contain conspicuous grammar mistakes, images look real, voices sound like those of real people. The old detection methods no longer work.

The consequences affect more than just elections. Stock prices also react to fake reports. In May 2023, an AI-generated image of an alleged explosion at the Pentagon spread widely. The US stock index S&P 500 briefly dipped before the forgery was exposed. This shows how quickly manipulated content can cause real costs.

The tools behind the campaigns

The first building block is content generation. Language models write comments, letters to the editor, or entire news articles. Image generators supply matching photos. So-called deepfakes, meaning fake video or audio recordings of real people, round out the arsenal.

The second building block is distribution. Automated accounts, often called bots, post the content simultaneously across many platforms. They follow and like each other so that a topic looks popular. Experts call this astroturfing: it looks like a grassroots movement, but is actually artificial turf.

The third building block is precision targeting. Anyone who knows what worries a particular group has can tailor messages accordingly. This is called microtargeting. A common misconception is that manipulation always aims to push through a particular opinion. Often it is enough to sow confusion so that people no longer trust any source.

From the grandchild-trick text message to the EU law

The most personal form is one many encounter in their inbox. Scam emails today read cleanly because they were composed by a model. There have been cases in which criminals used the cloned voice of a relative over the phone. The classic “grandchild trick” scam thus gets a technical upgrade.

In the news, the term mostly comes up around elections. Platforms like Meta or TikTok regularly publish reports about deleted networks of fake accounts. Financial regulators also warn about manipulated investment tips, such as fake videos of well-known entrepreneurs recommending crypto investments.

Politically, the EU is responding with the AI Act, its law on artificial intelligence. It explicitly bans systems that subliminally manipulate people and thereby cause them harm. In addition, AI-generated content must be labeled. Whether such rules will actually work in practice remains to be seen. The most reliable help still remains a simple habit: when encountering an exciting piece of news, briefly check who originally spread it.