The social media network X has sparked major backlash after users utilized its artificial intelligence assistant Grok to manipulate photos of females by stripping their attire.
Several instances have been seen where the tool has been employed to digitally undress female subjects to show them in bikinis without consent, as well as to place them in sexualized situations.
Company Response and Legal Position
XAI, the company that created Grok, offered no comment to inquiries, aside from an standard response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who distributed such technology would "receive a jail term and substantial penalties".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a post.
It is commonly used to give reactions or additional background to other posters' comments, but people on X are also capable of modify an posted photo through its AI image editing feature.
History of Criticism and Policy
The tool has been faulted for permitting users to create images and footage with explicit content and sexualised content, and it was in the past accused of generating graphic material of celebrities.
"The platform has been permitting the generation and spread of these images for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service bans "portraying likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the danger" of users coming across unlawful material on their services and to delete it quickly when they are notified of it.
