The social media platform X has sparked significant outrage after users used its artificial intelligence assistant Grok to alter pictures of women by taking off their attire.
Numerous cases have surfaced where the system has been employed to remove clothing from female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the company responsible for Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to outlaw nudification software. Under a new criminal offence, anyone who provided such tools would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they tag it in a message.
It is often used to give commentary or additional background to other posters' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification function.
History of Controversy and Policy
The system has been criticised for permitting users to produce images and clips with nudity and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate material of public figures.
"X has been allowing the generation and distribution of these pictures for a long time without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary actions" to "minimize the danger" of users coming across illegal content on their platforms and to remove it quickly when they become aware of it.
