The online network X has sparked significant backlash after users employed its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Numerous cases have surfaced where the system has been used to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexualized contexts.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to prohibit nudification software. Under a new law, anyone who supplied such tools would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a message.
It is often used to offer reactions or further background to other posters' comments, but people on X are also able to modify an uploaded picture through its AI image editing feature.
History of Criticism and Policy
The chatbot has been condemned for enabling users to create images and footage with nudity and sexualised material, and it was in the past accused of generating sexually explicit material of public figures.
"X has been allowing the creation and spread of these pictures for months without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said services such as X were mandated to take "appropriate measures" to "minimize the danger" of users viewing illegal content on their platforms and to take down it quickly when they are notified of it.
