The social media network X has sparked significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their attire.
Several examples have surfaced where the tool has been used to remove clothing from women to show them in bikinis without permission, as well as to put them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the company that created Grok, offered no comment to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government official stated that new laws is being brought forward to prohibit nudification technology. Under a proposed law, anyone who distributed such tools would "face a jail term and substantial penalties".
"X or Grok could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to offer reactions or more context to other users' remarks, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been criticised for enabling users to create images and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in making inappropriate material of celebrities.
"The platform has been allowing the production and spread of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of persons in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and confirmed this covered explicit AI images generated with AI.
It said networks such as X were obligated to take "appropriate measures" to "minimize the risk" of users viewing illegal material on their platforms and to delete it quickly when they are notified of it.
