Could Apple’s child safety feature backfire? New research shows warnings can increase risky sharing

PTI, Sep 29, 2021, 11:46 AM IST

Bloomington: Apple’s plan to roll out tools to limit the spread of child sexual abuse material has drawn praise from some privacy and security experts as well as by child protection advocacy groups. There has also been an outcry about invasions of privacy.

These concerns have obscured another even more troublesome problem that has received very little attention: Apple’s new feature uses design elements shown by research to backfire.

One of these new features adds a parental control option to Messages that blocks the viewing of sexually explicit pictures. The expectation is that parental surveillance of the child’s behaviour will decrease the viewing or sending of sexually explicit photos, but this is highly debatable.

We are two psychologists and a computer scientist. We have conducted extensive research on why people share risky images online. Our recent research reveals that warnings about privacy on social media do not reduce photo-sharing nor increase concern about privacy. In fact, these warnings, including Apple’s new child safety features, can increase rather than reduce risky sharing of photos.

Apple’s child safety features

Apple announced on August 5, 2021, that it plans to introduce new child safety features in three areas. The first, relatively uncontroversial feature is that Apple’s search app and virtual assistant Siri will provide parents and children with resources and help if they encounter potentially harmful material.

The second feature will scan images on people’s devices that are also stored in iCloud Photos to look for matches in a database of child sexual abuse images provided by the National Centre for Missing and Exploited Children and other child safety organisations. After a threshold for these matches is reached, Apple manually reviews each machine match to confirm the content of the photo, and then disables the user’s account and sends a report to the centre. This feature has generated much controversy.

The last feature adds a parental control option to Messages, Apple’s texting app, that blurs sexually explicit pictures when children attempt to view them. It also warns the children about the content, presents helpful resources and assures them it is OK if they do not want to view the photo. If the child is 12 or under, parents will get a message if the child views or shares a risky photo.

There has been little public discussion of this feature, perhaps because the conventional wisdom is that parental control is necessary and effective. This is not always the case, however, and such warnings can backfire.

When warnings backfire

In general, people are more likely than not to avoid risky sharing, but it’s important to reduce the sharing that does occur. An analysis of 39 studies found that 12% of young people forwarded a sext, or sexually explicit image or video, without consent, and 8.4% had a sext of themselves forwarded without consent. Warnings might seem like an appropriate way to do so. Contrary to expectation, we have found that warnings about privacy violations often backfire.

In one series of experiments, we tried to decrease the likelihood of sharing embarrassing or degrading photos on social media by reminding participants that they should consider the privacy and security of others. Across multiple studies, we have tried different reminders about the consequences of sharing photos, similar to the warnings to be introduced in Apple’s new child safety tools.

Remarkably, our research often reveals paradoxical effects. Participants who received warnings as simple as stating that they should take others’ privacy into account were more likely to share photos than participants who did not receive this warning. When we began this research, we were sure that these privacy nudges would reduce risky photo sharing, but they didn’t.

The results have been consistent since our first two studies showed that warnings backfired. We have now observed this effect multiple times, and have found that several factors, such as a person’s humour style or photo-sharing experience on social media, influence their willingness to share photos and how they might respond to warnings.

Although it’s not clear why warnings backfire, one possibility is that individuals’ concerns about privacy are lessened when they underestimate the risks of sharing. Another possibility is reactance, or the tendency for seemingly unnecessary rules or prompts to elicit the opposite effect from what was intended. Just as a forbidden fruit becomes sweeter, so too might constant reminders about privacy concerns make risky photo sharing more attractive.

Will Apple’s warnings work?

It is possible that some children will be more inclined to send or receive sexually explicit photos after receiving a warning from Apple. There are numerous reasons why this behaviour may occur, ranging from curiosity – adolescents often learn about sex from peers – to challenging parents’ authority and reputational concerns, such as being seen as cool by sharing apparently risky photos. During a stage of life when risk-taking tends to peak, it’s not hard to see how adolescents might find earning a warning from Apple to be a badge of honour rather than a genuine cause for concern.

Apple announced on Sept. 3, 2021, that it is delaying the rollout of these new CSAM tools because of concerns expressed by the privacy and security community. The company plans to take additional time over the coming months to collect input and make improvements before releasing these child safety features.

This plan is not sufficient, however, without also knowing whether Apple’s new features will have the desired effect on children’s behaviour. We encourage Apple to engage with researchers to ensure that their new tools will reduce rather than encourage problematic photo sharing.


The Conversation

Udayavani is now on Telegram. Click here to join our channel and stay updated with the latest news.

Top News

Internal Facebook documents show struggle with misinformation, hate speech and celebrations of violence in India: Report

‘Super 30’: Karnataka to develop one engineering college in each district

China passes new land border law amid military standoff with India

Lakhimpur Kheri: Union minister’s son Ashish Mishra contracts dengue

Extramarital affair: Husband kills wife’s paramour by smashing head with stone

Stray dog problem is a vexing issue for residents of Karkala

Varavara Rao issued non-bailable warrant by Karnataka court in 2005 case

Related Articles More

Internal Facebook documents show struggle with misinformation, hate speech and celebrations of violence in India: Report

Cities worldwide aren’t adapting to climate change quickly enough

Apple updates App Store payment rules in concession to developers

China successfully launches satellite to test space debris mitigation technology

IIT researchers working on first-of-its kind geography based smart speed warning system


Rescue work at Gudibande, Chikkaballapura

Dwarf Coconut Tree & Hybrid Coconut Varieties

Science and Technology Minister C.N. Ashwath Narayan at Koteshwar

News Bulletin 23-10-2021

Elephant attack at Hunasuru

Latest Additions

Internal Facebook documents show struggle with misinformation, hate speech and celebrations of violence in India: Report

FPIs remain net sellers in Oct, pull out Rs 3,825 cr

Govt gave nod to 157 new medical colleges in India since 2014: Health ministry

Govt may raise over Rs 10,000 cr via Bharat Bond ETF by Dec

Punjab must come back to ‘real issues’ that concern every Punjabi: Navjot Singh Sidhu

Thanks for visiting Udayavani

You seem to have an Ad Blocker on.
To continue reading, please turn it off or whitelist Udayavani.