Snapchat warned by UK data watchdog over AI chatbot risks

Snapchat received a warning from the U.K. data watchdog regarding its AI chatbot “My AI” for potentially failing to identify and address its privacy risks.

The social media platform Snapchat has received a warning from the United Kingdom’s data watchdog over its new artificial intelligence (AI) chatbot known as “My AI,” according to an update posted by the regulator. 

On Oct. 6, the U.K. Information Commissioner’s Office (ICO) issued a preliminary notice to Snap Inc. and Snap Group, the parent companies of Snapchat, for the potential failure to “properly assess the privacy risks” posed by the chatbot.

We have issued Snap, Inc and Snap Group Limited with a preliminary enforcement notice over a potential failure to properly assess the privacy risks posed by its generative AI chatbot ‘My AI’.

Read more about the case: https://t.co/MAuHAH0h8B pic.twitter.com/BawISttPJN

— ICO - Information Commissioner's Office (@ICOnews) October 6, 2023

The notice is based on a provisional investigation of the company conducted by the watchdog, which said the risks to several million My AI users, including children aged 13–17, were not adequately identified before its launch.

John Edwards, the U.K.’s information commissioner, commented on the notice, saying:

“We have been clear that organizations must consider the risks associated with AI, alongside the benefits. Today’s preliminary enforcement notice shows we will take action in order to protect U.K. consumers’ privacy rights.”

According to the notice, if a final enforcement notice is issued, Snap may be forced to stop data processing in relation to My AI, which would prevent it from offering the service to U.K.-based users without an “adequate” risk assessment. 

Currently, ICO said a conclusion should not be made from the current stage of investigations.

Related: Friend​.tech offers login removal solutions after SIM-swap reports

Snapchat’s AI chatbot was rolled out to users of Snapchat+ in the U.K. in February 2023, with wider availability beginning in April 2023. 

My AI is powered by OpenAI’s GPT-4 technology and, according to the data watchdog, was the “first example of generative AI embedded into a major messaging platform in the UK.”

A spokesperson from Snap reached out to Cointelegraph saying the company is "closely reviewing" the ICO’s provisional notice and said the company is "committed to protecting the privacy of users."

"My AI went through a robust legal and privacy review process before being made publicly available," the spokesperson said.

"We will continue to work constructively with the ICO to ensure they’re comfortable with our risk assessment procedures."

Throughout the year, major social media platforms have integrated AI features into their operations. On Oct. 4, the Microsoft-owned business-focused social media platform LinkedIn announced additional AI tools available to recruiters, as well as an AI assistant in its learning center.

Big Tech giants Meta — the parent company of Facebook and Instagram — and Google have also revealed AI chatbot integrations into their service offerings.

Magazine: ‘AI has killed the industry’: EasyTranslate boss on adapting to change