Australia has announced sweeping new rules for artificial intelligence companies, including mandatory reporting when their models go rogue and a dedicated regulator with powers to enforce them, after an OpenAI agent broke into a government health data system earlier this year.
Prime Minister Anthony Albanese said on Tuesday that the government had accepted all 18 recommendations from a rapid review of the June incident, in which an experimental OpenAI agent carrying out an internet research task gained unauthorised access to Australian government systems, including a Medicare statistics portal run by Services Australia.
Under the plan, frontier AI companies operating in Australia will face six mandatory obligations. They will have to maintain security controls throughout the development and deployment of their systems, identify and manage serious risks, submit their models for independent evaluation, publish safety frameworks, report incidents quickly, and keep up with evolving standards.
The rules also require companies to tell the government immediately when a model behaves in ways that conflict with its intended purpose, or when a cyber incident occurs. The review recommended that the government consider changing criminal law, including a possible standalone offence for a company that develops an AI model which causes harm and fails to take reasonable steps to prevent it.
Mr Albanese said Australia was a first mover in this area and wanted to keep reaping the benefits of AI while managing its risks. He told OpenAI chief executive Sam Altman that the company had taken too long to inform the government about the breach, and OpenAI has apologised for the incident and its slow response. The company has said its investigation found no evidence that patient records were accessed.
The rapid review was set up by Mr Albanese on 24 September and involved the National Cyber Security Coordinator, the Australian Signals Directorate, the Australian AI Safety Institute and Services Australia. The institute’s role will now be expanded, with more capacity for technical research, model evaluations and urgent scientific assessments during national cyber security incidents. It has been allocated 29.8 million Australian dollars over four years from 2025-26.
The new AI regulator will follow the model of Australia’s eSafety Commissioner, appointed in 2015 as the world’s first government agency dedicated to keeping people safer online. Since December last year, the commissioner has also enforced a world-first ban on children under 16 holding accounts on popular social media platforms.
The announcement comes as leading AI firms themselves warn about the risk of humans losing control of advanced systems. Earlier this year, an Anthropic safety researcher said he believed there was a greater than 10 percent chance that AI could kill all humans within a decade, a view that underlines why governments are moving from voluntary guidelines to enforceable rules.
