EU enforces new AI Act requiring deepfakes to be clearly marked as fake

From this Sunday, AI companies will be subject to new obligations under the EU’s AI Act requiring them to make deepfakes and other AI-generated content clearly recognisable as artificial.

However, technical challenges raise doubts about how effective the rules will be.

Two years ago, the EU adopted sweeping rules to regulate artificial intelligence and created a dedicated AI Office to oversee the landmark legislation’s implementation.

Among its many provisions, the AI Act includes the world’s first transparency regimes for synthetic media, requiring providers of generative AI systems like Anthropic’s Claude and OpenAI’s ChatGPT to ensure that artificially generated or manipulated content can be detected as such.

The rules are stricter for deepfakes—realistic AI-generated or AI-manipulated images, audio or videos that imitate a real person’s appearance, voice or actions in a way that could lead people to believe they said or did something that never happened.

From satire to manipulation

Like any technology, AI can be used for both creative and harmful purposes.

Brussels recognises that not all synthetic content is deceptive: content created for clearly artistic, creative, satirical or fictional purposes generally falls outside the scope of the deepfake disclosure requirements.

A famous example came in 2023, when an AI-generated image of Pope Francis wearing a fashionable white puffer jacket went viral worldwide. The image was fake, but it was largely understood as a joke.

The same technology, however, can also be turned to far more harmful ends.

In 2024, global pop star Taylor Swift became the target of a wave of AI-generated explicit images that spread online without her consent, fuelling renewed debate about the risks of synthetic media.

Deepfakes have also emerged as a tool of political manipulation.

In 2022, shortly after Russia launched its full-scale invasion of Ukraine, a fake video appeared showing Ukrainian President Volodymyr Zelenskyy telling soldiers to surrender. It was quickly debunked, but it demonstrated how synthetic media could be weaponised in wartime.

The concern is that increasingly realistic AI-generated content, amplified by social media platforms, could make it harder for citizens to tell genuine information from fabricated material.

Regulating deepfakes

To address the challenge, the European Commission developed a voluntary code of practice as a framework indicating how companies should disclose the artificial origin of synthetic content through machine-readable markings, as well as visible labels for deepfakes.

The idea is simple: people should know when they’re looking at content generated or altered by AI. That is why the rules apply to both the companies developing the technology and those using AI in a professional capacity, as personal uses are excluded from the scope.

The code of practice not only addresses the marking aspect, but also the associated detection mechanism that requires collaboration for all the actors involved: AI companies, social media platforms, civil society organisations and fact-checkers.

The harder question is whether the technology can reliably deliver on that promise.

One rule, many hurdles

The first challenge is geographical. The EU can regulate companies operating in or targeting the European market, but online content doesn’t stop at borders. A deepfake created outside Europe can still reach millions of European users within minutes.

The second challenge is technical maturity. Major AI companies like OpenAI and Google have broadly backed transparency requirements, signing the Commission’s code of practice. However, they have also warned that detection and marking technologies remain a moving target.

There is currently no single industry-wide solution. Companies are experimenting with different approaches — watermarking, metadata, content provenance systems — that don’t always work together nor talk to each other.

The third challenge is that digital traces are fragile. Experts warn that watermarks can be removed, altered or lost when content is edited, compressed or shared across platforms. Tracing the full history of an AI-generated image or video through multiple edits can be extremely difficult.

Because of these limitations, researchers argue that no single technology will be enough. Companies may instead need a multilayer approach combining several methods.

“A combination of solutions is more powerful than any single measure. Therefore, a best practice is to implement multiple layers of marking and detection for AI-generated content,” the European Commission’s technical study on the topic concluded.

Keeping the pace

The EU has taken a leading role in tackling one of the defining challenges of the AI era: a world in which synthetic content is increasingly difficult to distinguish from reality.

But whether Brussels can keep pace with a technology evolving at extraordinary speed will determine whether the AI Act’s transparency rules become an effective safeguard, or merely a label stuck on a problem that keeps outrunning it.