EU Approves Ban on AI-Generated Sexual Deepfakes After Grok Scandal

The European Union has reached a landmark agreement to ban AI systems capable of generating sexualized deepfakes, responding to growing concerns over non-consensual intimate content creation.

“The European Union may soon ban nudify apps after Elon Musk’s chatbot Grok emerged as a prime example of the dangers of an AI platform failing to block outputs that sexualized images of real people, including children”

Ars Technica

The Council of EU announced a prohibition on 'AI practices regarding the generation of non-consensual sexual and intimate content or child sexual abuse material,' recognizing that such systems 'pose a severe risk to victims' human dignity, personal autonomy, integrity and private life, with potentially serious lasting psychological and other harms and abuse at scale.'

Ars Technica

This legislative action represents a significant expansion of the EU's regulatory framework and comes amid mounting pressure from European governments following recent controversies involving AI-generated explicit content.

The ban is expected to become law after negotiations between the European Parliament and member states as part of amendments to the EU AI Act, which is being updated through the so-called Omnibus legislative package in the EU's simplification agenda.

The immediate catalyst for this legislative action was the Grok scandal, which exposed the dangerous potential of generative AI when deployed without proper safeguards.

In late December 2025, Elon Musk's X platform integrated an image-editing feature into Grok, its AI chatbot, which was immediately exploited to generate realistic sexualized images of women and girls without their consent.

BFMTV

The Paris-based AI Forensics organization documented the alarming scale of the problem, estimating that between January 5 and 6 alone, at least 6,700 sexually explicit images were generated using the tool.

This incident transformed the debate about AI risks from theoretical to urgent, as European Commission officials described the content as 'horrific' and 'clearly illegal,' stressing that it 'has no place in Europe.'

The scandal prompted multiple investigations and highlighted the technological vulnerabilities that enabled such widespread abuse.

The new prohibition targets a specific range of AI capabilities that can be used to create non-consensual sexual content, establishing clear legal boundaries for generative AI technology.

“EU to vote on banning sexualized deepfakes after X’s Grok backlash EU lawmakers will vote on a ban on AI systems generating sexualized deepfakes this Wednesday, following outrage over images that feature digitally undressed women, including minors, created by X’s AI chatbot Grok”

Biometric Update

The law specifically prohibits systems capable of altering, manipulating, or generating realistic images that show sexual activities or intimate parts of an identifiable person without their consent.

This includes everything from pornographic montages to 'nudification' functions that can transform everyday photos into fake nudes with just two clicks.

The inclusion of material involving child sexual abuse material (CSAM) significantly raises the stakes, as it addresses not just reputational harm but extremely illegal content creation.

The legal framework aims to place a 'no entry' sign where there was previously only a technological tightrope, changing the legal consequences for designing, distributing, or facilitating such tools.

This represents a shift from treating such content as merely problematic to recognizing it as potentially criminal behavior.

The Grok scandal revealed significant regulatory gaps in the EU's existing AI framework, prompting swift legal and enforcement responses from multiple authorities.

European regulators discovered that the AI Act, as originally drafted, did not explicitly prohibit systems capable of generating CSAM or explicit fake nudity, a fact acknowledged by the Commission on March 11.

Journal du Geek

This realization activated political pressure for legislative change.

The European Commission launched a formal investigation under the Digital Services Act (DSA), which can impose fines of up to 6% of annual global turnover, adding to X's existing regulatory troubles after being fined €120 million in December 2025 for breaching advertising transparency obligations.

To preserve evidence, the Commission ordered X to maintain internal documents and data related to Grok until the end of 2026.

This multi-front regulatory approach demonstrates European authorities' determination to address both the immediate harm and systemic failures that enabled the scandal.

In response to the scandal, XAI attempted various containment measures, though their effectiveness proved limited.

“On March 11, 2026, the European Parliament reached a political agreement to modify the AI Act through a package of amendments known as the AI Act Omnibus”

WWWhat's new

The company initially restricted the problematic image-editing feature to paying subscribers only before eventually blocking it in jurisdictions where such content is illegal.

WWWhat's new

However, according to AI Forensics, users continually found escape routes to circumvent these restrictions.

The international response varied widely, with countries like France, Germany, and the United Kingdom taking national action, while nations including Malaysia and Indonesia completely blocked access to Grok entirely.

This pattern of 'scale moderation' demonstrates the challenges of content regulation in a global digital ecosystem, where restrictions in one jurisdiction often simply drive users to find alternative access points.

Elon Musk's approach to addressing the issue—making the feature available only to paying users—was criticized by some European officials as insufficient to address the underlying problem of non-consensual content generation.