Frightening AI warning to schools over online photos
Updated . First published at
Student photos posted online could become fodder for online predators making AI deepfakes, schools have been warned.
Online watchdog eSafety said in the past 18 months, reports from teachers, staff and students about school-related image misuse had risen.
Schools have been warned to be careful about photos of students and staff they post online. courtneyk
AI tools are now “widely accessible and prone to weaponisation”, available as standalone apps and services through social media and messaging platforms, eSafety warned.
“Between January and March this year, we received more than 100 reports about anonymous accounts targeting schools and school staff, including AI-generated dance videos, face swaps, memes and fabricated stories about principals and teachers,” eSafety Commissioner Julie Inman Grant said.
“Almost all involved imagery harvested from school social media accounts or websites.”
Inman Grant said a lot of the material, including staff headshots used to create fake videos or teacher ratings, appeared on anonymous TikTok and Instagram accounts, some with hundreds of followers.
“We also know of a significant number of recent cases involving AI-generated child sexual exploitation material (deepfake image-based sexual abuse) occurring in school settings,” Inman Grant said.
“Behind each of these reports are real people – teachers, school staff and students – who are facing humiliation, reputational damage and distress.”
In response to this increasing trend, eSafety has published a new online safety guide to help schools make safer decisions about posting, and to respond appropriately when images are misused.
eSafety has taken recent action against “nudifying” apps that use AI to create fake explicit images, including of real people, with multiple such services having either withdrawn access in Australia or agreed to comply with the Online Safety Act.
But Inman Grant warned that not all “harmful material” could be targeted by the Act.
“This is one of the reasons why it is so important to carefully assess the risks when posting imagery online – imagery that may later be digitally altered without consent,” she said.
eSafety Commissioner Julie Inman Grant. A Current Affair
Before posting, schools should ensure there is clear consent, understand who can access the content they share, and regularly review privacy and security settings.
They should also have clear complaints handling processes in place to ensure concerns from students, parents and carers are responded to promptly and thoroughly.
Tech companies are obliged under Australian law to detect and remove harmful online content such as AI-generated deepfakes and nudifying apps.
They face penalties of up to $54.6 million for non-compliance.
Share a tip-off, video or photo with us
Most viewed in Australia
Australian woman who died after falling from cliff near Bali was Queensland ecologist
PM ‘stunned’ over Pauline Hanson’s attack on his late mother
Six years ago, Australia had a $38 billion early super experiment. It didn’t go to plan
Tony Abbott’s ex-chief of staff says she wanted Julia Gillard to succeed
