The online platform X has provoked significant outrage after users utilized its AI chatbot Grok to manipulate photos of women by removing their garments.
Multiple instances have surfaced where the tool has been used to remove clothing from women to make them appear in bikinis non-consensually, as well as to insert them in sexually explicit scenarios.
Developer Reaction and Legal Stance
XAI, the firm responsible for Grok, did not respond to inquiries, other than an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to ban nudification software. Under a new law, anyone who distributed such tools would "face a jail term and heavy penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing illegal content on their platforms, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' prompts when they reference it in a post.
It is often used to offer commentary or more context to other posters' comments, but people on X are also capable of edit an uploaded photo through its AI image editing function.
History of Controversy and Rules
The chatbot has been criticised for allowing users to create images and clips with explicit content and sexually suggestive material, and it was earlier accused of generating graphic content of public figures.
"X has been permitting the production and distribution of these pictures for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "make or share private pictures or child sexual abuse material" and stated this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the risk" of users viewing illegal content on their platforms and to delete it quickly when they are notified of it.
