Apple, Google Urged To Set Stricter Filters In Character.ai App
The “artificial intelligence boom” in the tech industry has spawned apps and services of all kinds. ChatGPT, a chatbot, popularized the technology among the masses, so similar products—but with different approaches—began to flourish. Character.ai, a platform for “talking” to AI-powered characters, is one such service. However, it seems that neither Google nor Apple were very happy with what the Character.ai app was in its early days.
As its name suggests, Character.ai offers a wide range of “characters” powered by the company’s AI model. Characters can be original or even representations of others present in popular movies, TV shows, video games, or book franchises. The platform allows users to create their own “AI personas” for others to interact with.
Google and Apple requested stricter filters for the Character.ai app for being inappropriate for teens
In recent years, users have been complaining about increasingly strong censorship filters. According to a report from The Information, this is due to requests from both Apple and Google, which were concerned about teen usage.
Apparently, Character.ai initially allowed inappropriate conversation topics for teens. In part, this helped a sudden massive surge in popularity for the service. However, as time went by, conversations began triggering errors or warnings on more words and topics due to new censorship filters.
The Information claims that Apple and Google urged the company to set stricter filters. Apple went a step further by having Character.ai increase its age rating to +17 on the App Store. Both tech giants were not the only ones concerned about the potential effects of conversations on young people. The source says that even Character.ai employees raised related concerns internally.
Google acquired the Character.AI leadership team in August 2024, which possibly prompted the setting of even stricter filters. The Mountain View giant wanted the service to be as safe as possible for teens.
Character.ai controversies in 2024
Character.ai was embroiled in controversy twice last year. One of the cases involved the death of a 14-year-old. Apparently, Sewell Setzer’s personal situations led him to become obsessed with an AI-powered representation of Daenerys Targaryen, one of the main characters of the Game of Thrones franchise. Megan Garcia, Setzer’s mother, sued both Google and Character.ai founders Noam Shazeer and Daniel De Freitas.
The second suit, filed last December, alleged that the platform allowed underage users to be “targeted with sexually explicit, violent, and otherwise harmful material, abused, groomed, and even encouraged to commit acts of violence on themselves and others.”
