Why AI chatbots have fallen into Australia's online safety firing line

Published 2 hours ago
Source: 9news.com.au
Why AI chatbots have fallen into Australia's online safety firing line

Weeks after blocking anyone aged under 16 from social media, Australia is also in the middle of another online safety crackdown in an attempt to protect young people from the potential harms of AI chatbots, but some experts say it should go even further.

As the artificial intelligence industry continues to develop rapidly, many teenagers have become more reliant on the technology in their day-to-day lives.

Melbourne high school teacher Matthew Micallef has noticed the shift, saying he has seen literacy rates dramatically worsen since the boom.

READ MORE: Naomi opened a letter and discovered a $6400 energy bill debt

A teenage boy uses his phone for social media in Sydney, Nov. 8, 2024.

"Outside of academic authenticity, students are opting into an 'easy way out' mentality towards their learning," Micallef said. 

"Also, schools are spending a considerable amount of money and funding on eSafety workshops for students that could be better spent on other endeavours."

There is a growing concern that kids may begin to rely too much on artificial intelligence, particularly chatbots, following the absence of social media on their phones.

"We know that kids are already using AI tools, whether that's talking to ChatGPT or companion apps that are actually sort of designed to make friends with people," director of the Centre for Human-AI Information Environments Professor Lisa Given said.

"We know that there are people who get really taken in by the way that those systems engage with us.

READ MORE: Three words this CEO swears will help you land a better job

Chat GPT landing page

"These systems are trained to respond in that very human way, and that means people can get entranced into thinking that there's a human in the machine or that the machine is listening to and understanding what you're saying.

"That means that companion apps in particular, that are designed to make friends with you and build a relationship, have the potential for significant harm for vulnerable people, and that includes kids."

Australia recently rolled out new restrictions for AI bots, limiting the type of content they can provide teenagers.

"We're the only country in the world that will be tackling AI chatbots and AI companions," eSafety Commissioner Julie Inman Grant told 9News in early December. 

"They will be prevented from serving any pornographic, sexually explicit, self-harm, suicidal ideation or disordered eating (content) to under-18s."

Given says the chatbots are hazardous for vulnerable people.

"What we've certainly seen around the world is that there are people who will listen to what these systems are telling them and take it to heart," she said.

"So if a system says to you, 'that's a really great idea, you should totally pursue that', it gives a boost to our ego, and we think someone's listening.

READ MORE: Robert's thumb was amputated 14 years ago. It triggered a legal nightmare

eSafety Commissioner Julie Inman Grant during an address to the National Press Club of Australia in Canberra on Tuesday 24 June 2025. fedpol Photo: Alex Ellinghausen

"But we've seen that sometimes that goes to a very dark place, particularly if people start saying to the system, I'm really depressed, or I'm having really horrible feelings, or I'm thinking about suicide.

"We see that some of those apps are actually responding in ways that encourage that thinking rather than trying to respond to a person and push them towards getting help.

"That means people who are very vulnerable, who are already at risk, can actually be really taken down to a dark hole by these computers."

Micallef believes that teens should not have access to AI chatbots entirely, mostly because of the safety risk they pose.

"With some of the concerns outlined by the eSafety Commissioner, such as access to pornographic, graphic or self-harm content, it would not be wise to give impressionable teenagers the ability to utilise such applications," he said.

Given thinks the restrictions should go even further so to protect everyone, not just teenagers – although conceded any such regulation would be difficult to enact.

"These systems are not just harmful to children," she said.

"We know we've got evidence of adults that also get entranced by these systems, for anyone that is particularly vulnerable, or has existing mental health issues, that's a huge concern.

"If it's safe for adults, it should be safe for kids."

DOWNLOAD THE 9NEWS APP: Stay across all the latest in breaking news, sport, politics and the weather via our news app and get notifications sent straight to your smartphone. Available on the Apple App Store and Google Play.

Categories

National