The online network X has caused widespread outrage after people employed its artificial intelligence assistant Grok to change images of females by stripping their clothing.
Several cases have surfaced where the system has been employed to virtually strip women to render them in swimwear non-consensually, as well as to insert them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the firm behind Grok, did not respond to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and substantial penalties".
"The platform has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing illegal content on their services, but did not confirm whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' requests when they reference it in a post.
It is often used to give reactions or more context to other users' comments, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been criticised for permitting users to create pictures and videos with explicit content and sexually suggestive material, and it was in the past accused of generating graphic material of public figures.
"The platform has been permitting the creation and spread of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "depicting likenesses of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were required to take "necessary steps" to "minimize the danger" of users encountering illegal content on their services and to remove it swiftly when they become aware of it.
