The Australian government has established the AI Safety Institute under its National AI Plan. Charlton described it as “a national testing capability” and provided the most detailed public account yet of the work it will do. But will it be enough to keep Australians safe from what could be one of the most powerful, rapidly advancing technologies the world has known?
Tue 14 Jul 2026 10.03 AEST
Photo: AAP Image/Rounak Amini
This week, Andrew Charlton, the federal assistant minister for science and technology, issued a stark warning about artificial intelligence (AI).
Speaking at the AI Safety Forum at the University of Sydney, he said powerful AI models “are already doing things their creators never intended: cheating, deceiving, going their own way”.
In response, the Australian government has established the AI Safety Institute under its National AI Plan. Charlton described it as “a national testing capability” and provided the most detailed public account yet of the work it will do.
But will it be enough to keep Australians safe from what could be one of the most powerful, rapidly advancing technologies the world has known?
AI’s risks and harms are already affecting Australians.
There are the problems we are already seeing today: nudify apps, more sophisticated scams and deepfakes, voice cloning, chatbots that have isolated teens and encouraged harms, and heightened cybersecurity risks that are alarming our security agencies.
Beyond immediate and present harms, experts are sounding the alarm about what’s coming. This includes rapidly expanding capabilities and agentic AI that we don’t yet have controls for.
In a report published last week, the United Nations’ Independent International Scientific Panel on AI also raised concerns about global concentration of power, resources, AI capability, inequality, and the technology’s impact on how we think, reason and work.
The AI Safety Institute has three broad tasks, as Andrew Charlton explained at this week’s forum. First, analysing and testing new models. Second, supporting government regulators and agencies to respond to emerging AI capabilities, risks, harms and trends. Third, shaping safe AI development, deployment and international governance in Australia’s interests.
It is already doing work on multi-agent risks with the Gradient Institute).
You’ve probably already heard about AI agents that can book your table at a restaurant or plan your travel. The basic idea of an AI agent is an AI system that can plan and take action with limited supervision (for longer and longer periods too). When AI agents interact with each other without supervision, unpredictable risks can arise.
The AI Safety Institute is also working with the CSIRO on AI alignment – the science of trying to ensure that AI systems (and AI agents) act in ways that are aligned with their users’ values and goals. This matters because at this stage, we cannot fully predict how AI will act. There are plenty of experiments in which AI has deceived users or avoided human-imposed controls, even detecting (and underperforming) when it is being tested.
The head of the AI Safety Institute, philosopher and Royal Australian Air Force reservist Kate Conroy, also spoke at this week’s forum. She outlined plans to take on both immediate harms affecting Australians, with attention to those affecting the most marginalised and vulnerable, as well as frontier risks and harms.
The creation of the institute is a critical step towards Australia taking the impacts of AI seriously.
But given the speed of AI development, and the range of issues that are emerging, it would be fair to ask whether its budget of A$29.4 million over four years is enough.
By contrast, the United Kingdom allocated around A$460 million in its 2025 spending review to the its AI Security Institute. Singapore’s AI Safety Institute has an annual budget of around A$11 million a year. Canada has allocated around A$50 million over five years to its AI Safety Institute.
All these budgets look insignificant next to industry spending. OpenAI alone sunk US$19 billion into research and development in 2025.
Proposals that the Australian institute’s budget should be significantly higher were strongly supported at this week’s AI Safety Forum.
The unequal impacts of AI and its huge potential to increase inequality, as highlighted by the UN’s new report, further highlight the fact there is much more work to do.
Two promising next steps would be to legislate a digital duty of care, to help address some AI-supercharged online harms, and to address digital exclusion to ensure all Australians can benefit from AI.
The Tech Policy Design Institute’s recent assessment also highlights Australia’s strengths but also key needs in areas such as securing computing power for research and public interest activities. This would ensure Australia can maintain its position in research, and that the public sector and civil society don’t fall behind the rest of the world.
The conversation about AI, the opportunities it presents for society, and the impacts it will have needs to be much broader. We must consider a wider set of actions if we’re going to really address Australia’s low levels of trust in AI and keep people safe from the technology’s many potential harms.
With reports of a change in approach from the government, perhaps that’s coming.
As multiple speakers highlighted at the AI Safety Forum, Australia has a perhaps unique opportunity to make a difference. We have extraordinarily skilled AI safety researchers as well as people working on social impacts, as well as a societal and moral imperative to address the risks and opportunities of AI for all Australians.
Kimberlee Weatherall, Professor of Law, University of Sydney
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Advertisement
The Northern Territory receives lots of funding from the rest of Australia, and for good reason. But, when it comes to transparency and accountability around how we spend your money, things aren’t so clear.
The Albanese government is facing fresh pressure over gambling reform after Australians lost more than $104 billion in the three years since being handed the Murphy review, while the Greens and the Coalition have forced its proposed legislation to a Senate inquiry.
