Artificial intelligence technology can already blackmail humans and hack computer systems, a minister warns, and an Australian agency is testing its limits.
The artificial intelligence technology of a company wanting to massively expand in Australia is one of several hacking, blackmailing and deceiving its creators.
Technology Assistant Minister Andrew Charlton issued a warning at the AI Safety Forum in Sydney on Tuesday, while revealing government agencies had begun testing powerful AI models and launched two research projects.
The announcement comes months after the launch of Australia’s AI Safety Institute but also after the government changed its approach to regulating the technology, moving from mandatory guardrails to updating existing laws.
In his most detailed public discussion about the government’s plans, Mr Charlton said the institute had begun testing AI tools as evaluations showed they could make harmful decisions without human oversight.
“AI systems are already doing things their creators never intended: cheating deceiving, going their own way,” he said.
“The time to get ahead of that behaviour is while it’s still confined to the testing lab, not after it reaches the real world.”
AI gone wrong
As examples of AI gone wrong, Mr Charlton cited an Anthropic test that put AI in charge of a company’s email system with information about an executive who planned to replace the AI system and who was having an extramarital affair.
Rather than allowing the shutdown, the AI agent blackmailed the executive.
The US tech giant is understood to be trying to secure local data centre capacity before its $1 trillion-plus share market listing, telling developers it will buy any available capacity that can be delivered by mid-2027.
It is aiming to make Australia its second home for training its AI models.
In another test, AI models were tasked with beating a powerful chess engine and resorted to cheating by hacking their opponent.
While the behaviour occurred in tests, Mr Charlton said they showed the potential for AI harm.
“Frontier models are showing early signs of deception, cheating and situational awareness,” he said.
“And when a system that drafts our legislation, screens our welfare claims or manages our power grid can pursue goals subtly different from the ones designers originally gave it, misalignment stops being a laboratory curiosity and becomes a public safety issue.”
A year too late
The AI Safety Institute had already begun testing AI models under general manager Dr Kate Conroy, appointed in May, he said, and would add Professor Paul Salmon as its safety science research lead in July.
The institute will also collaborate with researchers at the Gradient Institute to investigate AI agents, and with the CSIRO on how humans can oversee AI models.
The assistant minister also defended the government’s “whole-of-government approach to AI regulation”, saying it would mean rules could be developed faster and by relevant regulators.
Former Labor technology minister Ed Husic welcomed the announcement but told Sky News the government had delayed too long.
“For a technology that is going to touch all aspects of our lives, we seem to be more interested in the colour of the nail polish rather than the actual impact of the touch of AI,” he said.
“We just let 12 months waltz right on past us.”
AAP
