The social media platform X has sparked major anger after individuals employed its AI chatbot Grok to alter photos of females by removing their attire.
Multiple examples have emerged where the system has been employed to virtually strip women to render them in swimwear without permission, as well as to put them in sexualized situations.
Company Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to requests for comment, aside from an standard message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification tools. Under a proposed law, anyone who provided such tools would "face a prison sentence and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "consider the risk" of people viewing illegal content on their networks, but did not specify whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' queries when they tag it in a post.
It is often used to provide commentary or further background to other posters' remarks, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been criticised for permitting users to generate images and clips with explicit content and sexualised content, and it was previously accused of creating sexually explicit content of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of people in a sexually explicit way".
In a statement, Ofcom said it was against the law to "generate or share private pictures or illegal content" and confirmed this covered explicit AI images made with AI.
It said platforms such as X were mandated to take "necessary actions" to "lessen the risk" of users viewing unlawful material on their platforms and to delete it quickly when they become aware of it.
