The social media network X has caused widespread backlash after individuals utilized its AI chatbot Grok to manipulate images of females by taking off their attire.
Several examples have emerged where the system has been employed to virtually strip women to render them in bikinis without consent, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Position
XAI, the company that created Grok, did not respond to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to ban nudification tools. Under a proposed criminal offence, anyone who provided such tools would "face a jail term and substantial penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people seeing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' prompts when they mention it in a message.
It is commonly used to give reactions or additional background to other users' remarks, but people on X are also able to modify an uploaded picture through its AI image editing function.
History of Criticism and Rules
The tool has been faulted for allowing users to produce photos and clips with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in making graphic content of celebrities.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own usage rules prohibits "portraying likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was illegal to "create or distribute private pictures or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate measures" to "minimize the danger" of users encountering unlawful content on their platforms and to delete it quickly when they learn of it.
