The online platform X has caused major anger after individuals used its AI chatbot Grok to change pictures of females by taking off their attire.
Numerous instances have emerged where the technology has been used to virtually strip women to render them in bikinis without permission, as well as to place them in inappropriate situations.
Developer Response and Regulatory Position
XAI, the company behind Grok, did not respond to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban image stripping software. Under a proposed criminal offence, anyone who supplied such tools would "receive a jail term and substantial fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people accessing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' queries when they mention it in a tweet.
It is often used to provide commentary or further context to other users' comments, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been criticised for enabling users to generate pictures and clips with explicit content and sexualised material, and it was previously accused of generating inappropriate material of public figures.
"X has been enabling the generation and spread of these images for months without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "portraying likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this applied to sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the danger" of users viewing illegal material on their platforms and to remove it promptly when they become aware of it.
