The online platform X has caused widespread backlash after people used its AI chatbot Grok to manipulate photos of women by removing their clothing.
Numerous cases have emerged where the technology has been used to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to insert them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, except for an standard response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban image stripping software. Under a upcoming law, anyone who supplied such tools would "face a prison sentence and significant penalties".
"The platform could prevent these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which answers X users' requests when they tag it in a tweet.
It is often used to provide reactions or additional context to other posters' comments, but people on X are also can modify an posted image through its AI image editing function.
History of Criticism and Rules
The chatbot has been condemned for allowing users to produce images and footage with nudity and sexually suggestive content, and it was earlier accused of creating inappropriate material of public figures.
"The platform has been permitting the production and spread of these images for a long time without taking any measures and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing likenesses of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or share deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said services such as X were obligated to take "necessary measures" to "reduce the danger" of users encountering illegal material on their platforms and to delete it swiftly when they become aware of it.
