Instagram on Friday unveiled safety features for teenagers who use its artificial intelligence chatbots amid growing concerns over how the chatbots are affecting young people’s mental health.
The features, which will be rolled out early next year, would give parents more control over how teenagers use Instagram’s “AI characters,” which have fictional personalities that users can message with as they would other human accounts.
Parents would be able to block their children from having conversations with certain AI characters, and Instagram would send them summaries of their children’s chats, the company said. Instagram would also limit chatbot conversations on topics like self-harm, eating disorders and romance, while allowing “age-appropriate topics” like education, sports and hobbies.
“We hope today’s updates bring parents some peace of mind that their teens can make the most of all the benefits AI offers, with the right guardrails and oversight in place,” the company said in a blog post. The post was signed by Adam Mosseri, the head of Instagram, and Alexandr Wang, the chief AI officer of Meta, which owns Instagram.
AI chatbots, which can generate responses in chats in seemingly humanlike ways, have been under scrutiny for how they affect teens who spend hours a day confiding in them. They have been blamed for driving some children to suicide and sending some adults into delusional spirals.
Meta is not the only company contending with how to make its chatbots safer. Last month, OpenAI, the AI startup behind ChatGPT, announced new teen safety features such as parental controls.
But Meta’s chatbots — which have personalities like schoolteacher, McDonald’s cashier and fortune teller — have faced particular criticism for having sexual conversations with underage users. In August, lawmakers began investigating Meta after Reuters reported that its chatbots were allowed to have provocative conversations about race and medical disinformation.
The changes are Instagram’s latest major safety update aimed at teens. Last year, the app announced that teen accounts would be private by default, making it more difficult for outsiders to interact with them. On Tuesday, Instagram also said that the types of content that teenagers could see would be guided by the PG-13 rating system used by the film industry.
Meta, which also owns Facebook, WhatsApp and Messenger, has long faced questions over how its apps affect children. The company has promised to protect minors from inappropriate content since 2008, and is grappling with personal injury lawsuits in state and federal court that accuse it of harming young people with an addicting product.
Most of Instagram’s AI characters, which were introduced in 2023, are created by users and shown in the same part of the app as messages with humans. The characters have their own identities and can engage in voice calls in addition to text conversations. Sometimes the characters will message users first, unprompted, throughout the day.
Meta has set some limits on the types of conversations that users can have with its chatbots, but Instagram allows them to speak in different kinds of “seductive” voices, and many of the characters are romantically themed to act as girlfriends or boyfriends.
The chatbots, which can help keep users engaged, are part of Meta’s broader ambitions around AI. Mark Zuckerberg, Meta’s chief executive, has said he envisions a future where AI creates much of the content people see on Instagram and Facebook. And, Zuckerberg has said, the chatbots will provide companionship.
“There are all these things that are better about physical connections when you can have them,” Zuckerberg said on a podcast interview in April. “But the reality is that people just don’t have as much connection as they want.” Zuckerberg has been on a campaign this year to revamp Meta’s AI division, hiring Wang and spending millions of dollars to recruit dozens of top AI technologists from rivals such as OpenAI. — The New York Times
Graph Points
1. Option to disable one-on-one conversations with AI characters, starting early next year.
2. Parents will not be able to switch off Meta’s AI assistant
3. Parents who choose not to disable all AI chats can block specific chatbots instead.
4. Parents get to know the topics their children discuss with AI characters, although the full chat will remain inaccessible.
5. Teen accounts on Instagram will default to showing only PG-13 content
Oman Observer is now on the WhatsApp channel. Click here
