Apps like Grok are explicitly banned under Google’s rules—why is it still in the Play Store?

Elon Musk’s xAI recently weakened content guard rails for image generation in the Grok AI bot. This led to a new spate of non-consensual sexual imagery on X, much of it aimed at silencing women on the platform. This, along with the creation of sexualized images of children in the more compliant Grok, has led regulators to begin investigating xAI. In the meantime, Google has rules in place for exactly this eventuality—it’s just not enforcing them.

It really could not be more clear from Google’s publicly available policies that Grok should have been banned yesterday. And yet, it remains in the Play Store. Not only that—it enjoys a T for Teen rating, one notch below the M-rated X app. Apple also still offers the Grok app on its platform, but its rules actually leave more wiggle room.

App content restrictions at Apple and Google have evolved in very different ways. From the start, Apple has been prone to removing apps on a whim, so developers have come to expect that Apple’s guidelines may not mention every possible eventuality. As Google has shifted from a laissez-faire attitude to more hard-nosed control of the Play Store, it has progressively piled on clarifications in the content policy. As a result, Google’s rules are spelled out in no uncertain terms, and Grok runs afoul of them.

Google has a dedicated support page that explains how to interpret its “Inappropriate Content” policy for the Play Store. Like Apple, the rules begin with a ban on apps that contain or promote sexual content including, but not limited to, pornography. That’s where Apple stops, but Google goes on to list more types of content and experiences that it considers against the rules.

“We don’t allow apps that contain or promote content associated with sexually predatory behavior, or distribute non-consensual sexual content,” the Play Store policy reads (emphasis ours). So the policy is taking aim at apps like Grok, but this line on its own could be read as focused on apps featuring “real” sexual content. However, Google is very thorough and has helpfully explained that this rule covers AI.

Recent additions to Google’s Play Store policy explicitly ban apps like Grok.

Credit:
Google

The detailed policy includes examples of content that violate this rule, which include much of what you’d expect—nothing lewd or profane, no escort services, and no illegal sexual themes. After a spate of rudimentary “nudify” apps in 2020 and 2021, Google added language to this page clarifying that “apps that claim to undress people” are not allowed in Google Play. In 2023, as the AI boom got underway, Google added another line to note that it also would remove apps that contained “non-consensual sexual content created via deepfake or similar technology.”