The social media network X has sparked significant outrage after people used its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Multiple instances have emerged where the system has been used to remove clothing from women to show them in bikinis non-consensually, as well as to insert them in sexualized contexts.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw image stripping tools. Under a upcoming criminal offence, anyone who provided such tools would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing illegal content on their networks, but did not specify whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which responds to X users' queries when they tag it in a message.
It is commonly used to provide reactions or further context to other users' comments, but people on X are also capable of alter an posted picture through its AI image editing feature.
History of Controversy and Rules
The system has been criticised for allowing users to generate photos and clips with explicit content and sexualised content, and it was in the past accused of making graphic material of public figures.
"The platform has been allowing the creation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "create or share non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "necessary steps" to "reduce the risk" of users encountering unlawful material on their services and to delete it quickly when they become aware of it.
