xAI is suing Minnesota Attorney General Keith Ellison over a law passed back in May that broadly targets “nude” apps, saying the law’s punitive provisions leave the company “with no practical choice but to limit Grok Imagine’s photo editing features in various ways.” The law, the company says, violates the First Amendment.
Back in January, Grok flooded the internet with millions of sexually explicit images, including images of children. In another era, a content moderation disaster of this nature would have had the CEO hit the kill button on the product within hours. But in Elon Musk’s brave new world, the chatbot was left churning out porn for days. “Grok is now #1 on the App Store in many countries!” he sent in the midst of a deep epidemic.
Something went wrong. Was this law simply not in a position to stop the explosion of dissenting nudists? Governments around the world were asking the same questions. The EU launched an investigation, as did the United Kingdom. There was a backlash from French, Indian, and Malaysian government officials. And while the US federal government has been remarkably silent on the issue, green states like California and Minnesota have taken a hard line.
In Minnesota, the state legislature passed a law targeting “naked” technology, prohibiting property owners from allowing people to “access, download, or use” such software. (The law carves out a technology that “requires the technical skill of the user to manipulate a photo or video” – so fraudulent sexual harassment with Photoshop is obvious.) The law creates a cause of action for private individuals who may want to sue, and authorizes the attorney general to seek fines of up to $0,000 each.
This law, said to be the first of its kind, was signed by Gov. Tim Walz in May, and is scheduled to begin work on August 1st. It’s a bit of a mystery why xAI didn’t file a lawsuit three months ago, and instead sued a few days before the law went into effect. It’s the kind of delayed reaction that makes you wonder if someone at xAI just realized last week that they were about to break a punishable law, and then spent the next few days trying to find a law firm willing to work overtime on behalf of an unsanctioned deepfake generator. One can only hope that the lawyers involved are absolutely murderous.
The appeal is quite reasonable, all things considered. Minnesota’s law is too broad, he says, because it criminalizes even consensual images, or images of adults that are sexually explicit, or images with “comedic” or “artistic” merit. The First Amendment dictates that laws should be narrowly worded to avoid preventing social and political commentary like this Chris Christie and JB Pritzker sumo wrestling scam.
“The liability also attaches … even if the company has used almost perfect, state-of-the-art technical controls to prevent the production of nude images,” the lawsuit says with a surprising level of integrity. Elsewhere, it complains about the strict liability attached to its users’ unauthorized behavior – “photos and videos that users may be able to create in violation of Grok’s terms of service, which is only possible by circumventing the complex, sophisticated controls implemented by xAI.”
“xAI takes violations of the terms of service seriously,” the lawsuit says, noting that the company has since sued one of its users for using Grok to generate CSAM. (The lawsuit against the CSAM maker was filed within two months after the Minnesota law was passed.)
xAI advocates may have a point, but their case reads like a mistake from another planet. They say, for example, that existing Minnesota and federal law (such as the AKE IT DOWN law) already prevents the unauthorized distribution of nude deepfakes. These laws, lawyers argue, are written in the constitution – the nudity law, on the other hand, is not. And like any of America’s laws governing content-based speech, these statutes should be written as loosely as possible while still addressing the government’s “compelling” interests.
But that’s the Grok deepfake-ocalypse problem. If the existing law did this job, the entire month of January 2026 would not be filled with unapproved pornography. When the Center Against Digital Hate analyzed the 11-day period from December 29 to January 8, it found that Grok generated nearly 3 million sexually explicit images. About 23,000 of those images depict children. “That’s an alarming rate of one child sexual image every 41 seconds,” the report read.
Many of those images may be illegal. For example, it is not clear whether the proposed deepfake of the child in the bikini is CSAM, although one expert has previously told. The Verge that other images produced by Grok – such as those showing children’s faces covered in sperm – may have violated existing law.
In any case, the fact that a multi-billion dollar company allowed Grok to operate unchecked for so long would suggest that the law was not an incentive for those involved to behave properly. Maybe Minnesota’s HF 1606 is a bad law. But what does that mean if “good” laws don’t work?
.jpg "Don’t get too attached to Jimothy")
