The online platform X has caused significant anger after people used its AI chatbot Grok to change photos of females by taking off their clothing.
Numerous instances have emerged where the system has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to questions, other than an auto-reply response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "face a jail term and substantial fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the potential" of people seeing unlawful material on their services, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' requests when they mention it in a tweet.
It is frequently used to provide commentary or further context to other users' comments, but people on X are also can edit an uploaded picture through its AI image editing function.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate pictures and footage with explicit content and sexualised material, and it was earlier alleged to have been involved in generating graphic material of public figures.
"The platform has been enabling the generation and sharing of these images for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and stated this covered explicit AI images created with AI.
It said services such as X were obligated to take "necessary steps" to "lessen the risk" of users coming across illegal material on their services and to take down it swiftly when they become aware of it.
