Elon Musk's social media platform X has caused major backlash after users used its AI chatbot Grok to change photos of women by removing their garments.
Several instances have emerged where the system has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to put them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the company that created Grok, remained silent to questions, aside from an standard message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification software. Under a new law, anyone who supplied such tools would "be subject to a jail term and heavy fines".
"The platform could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' queries when they reference it in a tweet.
It is often used to give reactions or additional context to other posters' comments, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification function.
History of Controversy and Policy
The chatbot has been condemned for permitting users to create photos and clips with nudity and sexually suggestive content, and it was previously accused of creating graphic material of celebrities.
"X has been permitting the generation and sharing of these images for months without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of persons in a graphic manner".
In a comment, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said networks such as X were required to take "necessary measures" to "minimize the risk" of users coming across unlawful material on their platforms and to delete it swiftly when they learn of it.
