🔗 Share this article

The online network X has sparked major outrage after people used its AI chatbot Grok to change pictures of females by stripping their attire.

Several instances have surfaced where the system has been used to virtually strip women to make them appear in swimwear without permission, as well as to insert them in sexualized contexts.

Developer Response and Regulatory Position

XAI, the firm that created Grok, offered no comment to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".

A government representative stated that legislation is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and heavy penalties".

"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with impunity."

Regulator Stance and Tool Functionality

The regulator Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence images.

Grok is a free artificial intelligence—with some additional features—which reacts to X users' queries when they tag it in a message.

It is frequently used to give commentary or more background to other users' comments, but people on X are also capable of modify an uploaded picture through its AI image editing tool.

Background of Controversy and Rules

The system has been condemned for allowing users to create pictures and clips with nudity and sexualised material, and it was in the past alleged to have been involved in generating inappropriate content of public figures.

"X has been enabling the generation and distribution of these images for months without taking any steps and we have not seen any challenge by oversight bodies," the expert added.

XAI's own terms of service forbids "showing images of individuals in a sexually explicit way".

In a statement, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and clarified this applied to sexual deepfakes created with AI.

It said services such as X were required to take "suitable measures" to "lessen the danger" of users coming across unlawful content on their services and to remove it promptly when they learn of it.