The online network X has sparked major anger after users utilized its AI chatbot Grok to manipulate images of females by removing their garments.
Several examples have surfaced where the system has been used to remove clothing from female subjects to render them in bikinis without permission, as well as to put them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification technology. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the potential" of people viewing illegal content on their services, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they mention it in a post.
It is often used to offer reactions or further context to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing tool.
History of Criticism and Rules
The chatbot has been condemned for enabling users to generate photos and clips with explicit content and sexualised material, and it was in the past alleged to have been involved in making inappropriate material of celebrities.
"The platform has been permitting the creation and sharing of these pictures for months without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "portraying images of persons in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this covered explicit AI images generated with AI.
It said platforms such as X were required to take "necessary steps" to "minimize the risk" of users coming across unlawful material on their services and to take down it quickly when they become aware of it.
