diff --git a/docs/preparing-for-agi/framework/governance.md b/docs/preparing-for-agi/framework/governance.md index f68a9cf..4ad9eb0 100644 --- a/docs/preparing-for-agi/framework/governance.md +++ b/docs/preparing-for-agi/framework/governance.md @@ -3,7 +3,7 @@ icon: lucide/scale title: "AI Governance - Laws and Institutions" description: "How laws, institutions and international cooperation can support accountability and safer advanced-AI deployment in Australia." keywords: "AI governance, AI regulation Australia, AI policy, AI transparency, AI accountability, international AI coordination, AI Safety Institute Australia, risk-based AI regulation" -last-reviewed: "2026-07-18" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_title: "AI Governance - Laws, Institutions and Coordination" og_description: "How laws and institutions shape safe AI development and deployment in Australia." @@ -114,9 +114,9 @@ Some AI capabilities and deployment practices are changing faster than governanc There is no consensus on the right governance model for frontier AI. Three broad approaches have distinct trade-offs. -**Compute governance** uses the physical infrastructure of AI (chips, data centres, training runs) as a regulatory lever. Compute is physical, measurable and concentrated in a small number of supply chains — making it a practical chokepoint for oversight. But algorithmic efficiency gains erode thresholds over time and emerging decentralised training methods could eventually route around regulated infrastructure (Sastry et al. [2024](https://arxiv.org/abs/2402.08797); Kryś, Sharma & Egan [2025](https://arxiv.org/abs/2501.02470)). +**Compute governance** uses the physical infrastructure of AI (chips, data centres, training runs) as a regulatory lever. Compute is physical, measurable and concentrated in a small number of supply chains — making it a practical chokepoint for oversight. But algorithmic efficiency gains erode thresholds over time and emerging decentralised training methods could eventually route around regulated infrastructure (Sastry et al. [2024](https://arxiv.org/abs/2402.08797); Kryś, Sharma & Egan [2025](https://arxiv.org/abs/2507.07765)). -**Private governance** combines lab self-regulation with third-party auditors (Ball [2025](https://arxiv.org/abs/2504.11501)). It is flexible but relies on voluntary compliance and a mature auditing ecosystem that may not yet exist — an assumption found in four of five major proposals reviewed by CSET ([Narayanan et al. 2025](https://cset.georgetown.edu/publication/ai-governance-at-the-frontier/)). +**Private governance** relies on privately run bodies, such as government-authorised certifiers (Ball [2025](https://arxiv.org/abs/2504.11501)), alongside lab commitments and third-party audits. It is flexible but relies on voluntary compliance and a mature auditing ecosystem that may not yet exist — an assumption found in four of five major proposals reviewed by CSET ([Narayanan et al. 2025](https://cset.georgetown.edu/publication/ai-governance-at-the-frontier/)). **Decentralised mechanisms** — distributed evaluation networks, community governance structures and cooperative ownership models — complement rather than replace the other approaches. See [Decentralisation](../decentralisation.md) for detailed analysis. @@ -159,8 +159,8 @@ The [US NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-mana **What this means for you:** -- Australia currently combines existing laws and sectoral regulation with voluntary AI guidance and specialist institutions. The [Guidance for AI Adoption (AI6)](../../safety-standards/guidance-for-ai-adoption-ai6.md), published in October 2025, is the primary responsible-adoption guidance; its six essential practices evolve and incorporate the earlier Voluntary AI Safety Standard. See [AI Standards & Legislation](../../safety-standards/index.md) for current status -- The [National AI Plan](../../safety-standards/ai-australian-legislation.md) (December 2025) sets out this approach. Australia's AI Safety Institute is now operating within the Department of Industry, Science and Resources (as at July 2026) +- Australia currently combines existing laws and sectoral regulation with voluntary AI guidance and specialist institutions. The [Guidance for AI Adoption (AI6)](../../safety-standards/guidance-for-ai-adoption-ai6.md), published in October 2025, is the primary responsible-adoption guidance; its six essential practices evolve the earlier Voluntary AI Safety Standard, which remains available. See [AI Standards & Legislation](../../safety-standards/index.md) for current status +- The [National AI Plan](../../safety-standards/ai-australian-legislation.md) (December 2025) sets out this approach. Australia's AI Safety Institute is now operating within the Department of Industry, Science and Resources (as at October 2026) - If you're in government procurement, you have immediate leverage: require safety standards before buying - For business, understanding emerging requirements now helps you prepare before they become mandatory @@ -181,7 +181,7 @@ AI cuts across health, finance, critical infrastructure, defence and consumer pr **Key institutional elements:** **Lead coordination body** -A body (possibly within Treasury, Home Affairs or as independent agency) that: +A body (Australia has started with an Office of AI in the Department of the Prime Minister and Cabinet, [established in July 2026](https://www.pm.gov.au/media/ai-australias-interests)) that: - Coordinates AI policy across government - Provides expertise and guidance to sectoral regulators @@ -203,7 +203,7 @@ Through Australia's AI Safety Institute, regulators, research organisations and - Provide independent assurance (not just trusting developer claims) - Support incident analysis and learning -METR's [Common Elements of Frontier AI Safety Policies](https://metr.org/common-elements) (2025) identifies nine recurring elements across major lab commitments: capability thresholds, model weight security, deployment mitigations, development and deployment halting conditions, full capability elicitation, evaluation timing and frequency, accountability and policy updates. This convergence could support a mandatory assurance standard. +METR's [Common Elements of Frontier AI Safety Policies](https://metr.org/common-elements) (December 2025 update) identifies nine recurring elements across major lab commitments: capability thresholds, model weight security, deployment mitigations, deployment halting conditions, development halting conditions, full capability elicitation, evaluation timing and frequency, accountability and policy updates. This convergence could support a mandatory assurance standard. **Research and horizon scanning** Funding and mandate for: @@ -240,6 +240,11 @@ For high-risk systems, require disclosure of where and why they are deployed, kn Require protected reporting of significant failures, harms and near-misses. Where appropriate, a public incident database can support collective learning and improved standards. +!!! note "Real-world signal: an AI agent incident in a government system (2026)" + In September 2026 the Prime Minister disclosed that an OpenAI agent, during OpenAI's internal testing, had gained unauthorised access on 18 June 2026 to a Medicare statistics portal run by Services Australia. OpenAI first notified the agency on 10 September by emailing a public inbox. ABC News then reported that the Government intends its planned AI standards legislation to require AI companies to report such incidents to the affected organisation and the Australian Signals Directorate. This is a reported policy intention, not law. The delay shows why clear, fast incident reporting duties matter for agentic AI. ([ABC News, 24 September 2026](https://www.abc.net.au/news/2026-09-24/ai-agent-accessed-australian-government-site-pm-says/107189078); [ABC News, 29 September 2026](https://www.abc.net.au/news/2026-09-29/openai-medicare-breach-fuels-tougher-approach-to-rogue-ai/107204948)) + + + **Audit rights** Regulators need powers to audit high-risk systems. Critical infrastructure operators need audit rights over suppliers, while independent researchers need appropriately safeguarded access for safety research. @@ -254,7 +259,7 @@ Governance frameworks should clarify liability for AI harms and provide accessib **What this means for you:** -Australian privacy law already includes a new automated decision-making transparency obligation that commences on 10 December 2026. Covered APP entities will need to describe in their privacy policies certain uses of personal information in substantially automated decisions that significantly affect individuals' rights or interests. Other transparency, audit and incident-reporting duties depend on the applicable law and sector. Regardless of whether reporting is legally required, establishing an internal AI incident process now builds useful practice and institutional knowledge. See the [OAIC's current guidance development](https://www.oaic.gov.au/engage-with-us/consultations/consultation-on-guidance-for-transparency-in-automated-decision-making) for scope and commencement details. +Australian privacy law already includes a new automated decision-making transparency obligation that commences on 10 December 2026. Covered APP entities will need to describe in their privacy policies certain uses of personal information in substantially automated decisions that significantly affect individuals' rights or interests. Other transparency, audit and incident-reporting duties depend on the applicable law and sector. Regardless of whether reporting is legally required, establishing an internal AI incident process now builds useful practice and institutional knowledge. The OAIC published its final guidance on 30 September 2026, including an [APP 1.7–1.9 transparency obligation fact sheet](https://www.oaic.gov.au/__data/assets/pdf_file/0021/269013/APP-1.7-1.9-Transparency-Obligation-Fact-Sheet.PDF) and updated APP 1 Guidelines. --- @@ -298,7 +303,7 @@ Cooperation with the US, UK and other Five Eyes partners, together with particip - Support development of international AI safety standards - Advocate for strong safety priorities, not just commercial interests -**International landscape (as at July 2026)** +**International landscape (as at October 2026)** The [International AI Safety Report](https://internationalaisafetyreport.org/) (2026) provides a shared evidence base on misuse and malfunction risks, but national governance approaches still differ. Australia should plan for cooperation across several regulatory and technical networks rather than assume that a single global regime will emerge. @@ -312,7 +317,7 @@ Partnerships with peer institutes and standards bodies can expand Australia's ac **What this means for you:** -Australia has strong relationships with key AI safety allies and is well-positioned to contribute to international coordination. If your organisation operates across borders, understanding emerging international standards helps you prepare. For policy makers, Australia's role may be convening, bridge-building and technical contribution rather than direct leverage over frontier labs—and advocating for our region (Pacific, Southeast Asia) in global governance. +Australia participates in international AI safety networks and has arrangements with several AI developers, giving it channels to contribute to international coordination. If your organisation operates across borders, understanding emerging international standards helps you prepare. For policy makers, Australia's role may be convening, bridge-building and technical contribution rather than direct leverage over frontier labs—and advocating for our region (Pacific, Southeast Asia) in global governance. --- diff --git a/docs/preparing-for-agi/timelines.md b/docs/preparing-for-agi/timelines.md index 874fcfe..faef794 100644 --- a/docs/preparing-for-agi/timelines.md +++ b/docs/preparing-for-agi/timelines.md @@ -3,7 +3,7 @@ icon: lucide/clock title: "AGI Timelines - When Could Advanced AI Arrive?" description: "Public estimates for advanced AI vary dramatically. Compare institutional positions, executive forecasts and researcher surveys to inform Australian planning." keywords: "AGI timeline, artificial general intelligence timeline, AI predictions, OpenAI predictions, Anthropic forecasts, DeepMind AGI, AI safety timeline Australia" -last-reviewed: "2026-07-18" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_description: "Compare institutional positions, executive forecasts and researcher surveys on advanced AI timelines." og_type: "article" @@ -40,7 +40,7 @@ Statements from frontier developers are useful because they reveal what those or **Published position:** Working toward AGI; no precise institutional arrival date - OpenAI's stated mission is to ensure AGI benefits humanity. In a [December 2025 reflection](https://openai.com/index/ten-years/), CEO Sam Altman wrote that the organisation had a clearer line of sight toward that mission and expressed a personal expectation that superintelligence would be built by 2035. This is an executive forecast, not a dated institutional commitment to deliver AGI. + OpenAI's stated mission is to ensure AGI benefits humanity. In a [December 2025 reflection](https://openai.com/index/ten-years/), CEO Sam Altman wrote that the organisation had a clearer line of sight toward that mission and expressed a personal expectation that superintelligence would be built by 2035. This is an executive forecast, not a dated institutional commitment to deliver AGI. In [September 2026](https://openai.com/index/research-acceleration-view-inside-openai/), OpenAI said it had reached its internal goal of an "automated research intern" and was making progress toward an automated AI researcher by March 2028. These are internal research milestones set by the company, not an AGI arrival date. - :material-brain:{ .lg .middle } **Anthropic (USA)** @@ -48,7 +48,9 @@ Statements from frontier developers are useful because they reveal what those or **Published expectation:** "Powerful AI" as soon as **late 2026 or early 2027** - In a [March 2025 policy submission](https://www.anthropic.com/news/anthropic-s-recommendations-ostp-u-s-ai-action-plan), Anthropic said it expected "powerful AI systems" in late 2026 or early 2027. It defined these as systems matching or exceeding Nobel-level expertise across most disciplines and able to perform digital work autonomously. That is a specific company expectation, but it is not identical to every definition of AGI. + In a [March 2025 post on its policy submission](https://www.anthropic.com/news/anthropic-s-recommendations-ostp-u-s-ai-action-plan), Anthropic said it expected "powerful AI systems" in late 2026 or early 2027. The submission itself was more cautious, saying such systems "could emerge as soon as late 2026 or 2027". It defined these as systems matching or exceeding Nobel-level expertise across most disciplines and able to perform digital work autonomously. That is a specific company expectation, but it is not identical to every definition of AGI. + + - :material-google:{ .lg .middle } **DeepMind (Google, USA/UK)** @@ -56,7 +58,7 @@ Statements from frontier developers are useful because they reveal what those or **Published position:** Building systems on a path toward AGI; no precise institutional arrival date - Google DeepMind says it is building systems "on the path" to AGI but does not publish a precise corporate arrival date. Co-founder Shane Legg's longstanding personal forecast of a 50% chance of human-level AI by 2028 is often cited, but it should not be presented as DeepMind's institutional forecast. See DeepMind's [March 2026 description of its AGI objective](https://deepmind.google/blog/10-years-of-alphago/). + Google DeepMind says it is building systems "on the path" to AGI but does not publish a precise corporate arrival date. Co-founder Shane Legg's longstanding personal forecast, which he now frames as a 50% chance of "minimal AGI" by 2028, is often cited, but it should not be presented as DeepMind's institutional forecast. See DeepMind's [March 2026 description of its AGI objective](https://deepmind.google/blog/10-years-of-alphago/). - :material-flag:{ .lg .middle } **Chinese Frontier Labs** @@ -66,13 +68,15 @@ Statements from frontier developers are useful because they reveal what those or Chinese developers including Baidu, Alibaba, Tencent, ByteDance and Zhipu AI are developing frontier models, alongside state-backed research institutes. Publicly available evidence supports strong strategic commitment and investment, but not a single reliable "China AGI by 2030" forecast. See the National Bureau of Asian Research's [2024 ecosystem overview](https://www.nbr.org/publication/chinas-generative-ai-ecosystem-in-2024-rising-investment-and-expectations/). + + The defensible conclusion is that several leading developers are actively preparing for much more capable systems. Their statements are relevant short-horizon scenarios, not consensus forecasts. ### Australian and Australia-connected perspectives -Australian perspectives span the same disagreement. **[Toby Walsh](https://www.unsw.edu.au/staff/toby-walsh)** (UNSW) has used 2062 as a longer planning horizon while arguing for sustained policy preparation rather than confidence in a precise arrival date. **Helen Toner**, who was born in Melbourne and later served on OpenAI's board, frames the disagreement through [three unresolved questions](https://helentoner.substack.com/p/unresolved-debates-about-the-future) about scaling, AI-assisted research and autonomous agents; those questions are used below to explain why forecasts diverge. +Australian perspectives span the same disagreement. **[Toby Walsh](https://www.unsw.edu.au/staff/toby-walsh)** (UNSW) said in 2018 that he considered 2062 the year AI would match human intelligence, while arguing for sustained policy preparation rather than confidence in a precise arrival date. **Helen Toner**, who was born in Melbourne and later served on OpenAI's board, frames the disagreement through [three unresolved questions](https://helentoner.substack.com/p/unresolved-debates-about-the-future) about scaling, AI-assisted research and autonomous agents; those questions are used below to explain why forecasts diverge. **Epoch AI** [reviews](https://epoch.ai/blog/literature-review-of-transformative-artificial-intelligence-timelines) published forecasting literature and maintains empirical datasets on model capabilities. **METR** measures the length of software tasks that frontier models can complete autonomously; its observed trend is a capability indicator, not a direct AGI arrival forecast. @@ -91,9 +95,9 @@ The [2023 survey of 2,778 AI researchers](https://aiimpacts.org/wp-content/uploa | Source | Published claim | Evidence type | Important limitation | |--------|-----------------|---------------|----------------------| | **OpenAI / Sam Altman** | Personal expectation of superintelligence by 2035 | Executive forecast (2025) | Not a probability or institutional delivery date | -| **Anthropic** | Powerful AI as soon as late 2026 or early 2027 | Company policy submission (2025) | Uses Anthropic's own capability definition | +| **Anthropic** | Powerful AI as soon as late 2026 or early 2027 | Company blog post on its policy submission (2025) | Uses Anthropic's own capability definition | | **Google DeepMind** | Explicitly working on a path toward AGI | Institutional objective (2026) | No precise corporate arrival date | -| **Shane Legg** | 50% chance of human-level AI by 2028 | Individual forecast | Not DeepMind's institutional forecast | +| **Shane Legg** | 50% chance of "minimal AGI" by 2028 | Individual forecast | Not DeepMind's institutional forecast | | **Aschenbrenner** | AGI around 2027 | Individual scenario analysis | Contested assumptions; not independent consensus | | **2023 researcher survey** | 50% aggregate probability by 2047 for machines outperforming humans in every task | Survey of 2,778 AI researchers | Definition differs from lab and economic-transformation concepts | @@ -103,7 +107,7 @@ The [2023 survey of 2,778 AI researchers](https://aiimpacts.org/wp-content/uploa ## Why do AGI timeline predictions vary so much? -**Helen Toner** (Australian AI policy researcher, former OpenAI board member, Director of Strategy at Georgetown CSET) identifies [three fundamental unresolved questions](https://helentoner.substack.com/p/unresolved-debates-about-the-future) that determine whether—and how soon—transformative AI will arrive: +**Helen Toner** (Australian AI policy researcher, former OpenAI board member, [Executive Director of Georgetown CSET](https://cset.georgetown.edu/staff/helen-toner/)) identifies [three fundamental unresolved questions](https://helentoner.substack.com/p/unresolved-debates-about-the-future) that determine whether—and how soon—transformative AI will arrive: !!! question "Three fundamental questions shaping AGI timelines" @@ -117,7 +121,7 @@ The [2023 survey of 2,778 AI researchers](https://aiimpacts.org/wp-content/uploa Beyond these three questions, five additional factors drive dramatic variation: -**Definition ambiguity:** "AGI" ranges from benchmark performance (arguably achieved) to economically transformative (automating most jobs) to true general intelligence (human-level across all domains) to superintelligence. Frontier labs use looser definitions; academics stricter ones. +**Definition ambiguity:** "AGI" ranges from benchmark performance on specific tests to economically transformative (automating most jobs) to true general intelligence (human-level across all domains) to superintelligence. Frontier labs use looser definitions; academics stricter ones. **Scaling vs. breakthroughs:** Frontier labs believe current approaches will scale to AGI (more compute + data + algorithms). Academic sceptics argue fundamental breakthroughs are needed for robust reasoning, understanding and transfer learning. Both could be right—scaled systems might automate jobs without solving all hard AI problems. @@ -145,7 +149,7 @@ Beyond these three questions, five additional factors drive dramatic variation: **Use as:** Urgent stress-test scenario | **Impact:** Transformational - If frontier labs are correct, Australia faces advanced AI transformation within one election cycle. This demands immediate capability building for AI evaluation and governance, regulatory frameworks in place before deployment, international coordination on safety standards and strategic decisions about critical infrastructure dependencies. + If frontier labs are correct, Australia could face advanced AI transformation within one or two election cycles. This demands immediate capability building for AI evaluation and governance, regulatory frameworks in place before deployment, international coordination on safety standards and strategic decisions about critical infrastructure dependencies. **Planning implication:** Build core capabilities now—evaluation teams, governance frameworks, international partnerships @@ -217,13 +221,13 @@ Rather than relying on predictions, monitor these concrete signals that timeline Frontier labs see internal capability gains before they're public. They have access to data external researchers don't. Their predictions reflect this privileged information. - But they also have incentives to appear cutting-edge for talent recruitment and investment. "AGI in 5 years" attracts resources, even if actual timelines are uncertain. + But they also have incentives to appear cutting-edge for talent recruitment and investment. Short timeline claims can attract investment and talent, even when actual timelines are uncertain. **Best approach:** Take frontier lab predictions seriously as lower-bound scenarios (they might be right) while maintaining scepticism about marketing-driven optimism. Plan for a range that includes their forecasts. ??? question "What if AGI never arrives? Is this all wasted effort?" - **Near-term advanced AI still creates most of the governance challenges:** + **Near-term advanced AI still creates many of the governance challenges:** Even without "true AGI," systems that automate most knowledge work transform employment, concentrate power and create security risks. Preparing for AGI means preparing for this transformation. @@ -245,32 +249,31 @@ Rather than relying on predictions, monitor these concrete signals that timeline **Frontier lab statements and thought leaders:** - OpenAI (2023) ["Planning for AGI and beyond"](https://openai.com/blog/planning-for-agi-and-beyond) — Official company perspective on AGI development - - Altman, Sam (2025) ["The Gentle Singularity"](https://blog.samaltman.com/the-gentle-singularity) — OpenAI CEO on gradual but transformative AGI arrival this decade + - Altman, Sam (2025) ["The Gentle Singularity"](https://blog.samaltman.com/the-gentle-singularity) — OpenAI CEO on a gradual path toward digital superintelligence, with expected milestones in 2026, 2027 and the 2030s - Amodei, Dario (2024) ["Machines of Loving Grace"](https://www.darioamodei.com/essay/machines-of-loving-grace) — Anthropic CEO on powerful AI potentially arriving as early as 2026 and its possible societal effects - Anthropic (March 2025) ["Recommendations to OSTP for the U.S. AI Action Plan"](https://www.anthropic.com/news/anthropic-s-recommendations-ostp-u-s-ai-action-plan) — Organisational expectation of powerful AI in late 2026 or early 2027 - Aschenbrenner, Leopold (2024) ["Situational Awareness: The Decade Ahead"](https://situational-awareness.ai/) — Former OpenAI researcher on AGI by 2027 and national security implications - - DeepMind research publications on AGI timelines and capabilities **Expert surveys and forecasting:** - - [Epoch AI Literature Review of TAI Timelines](https://epoch.ai/blog/literature-review-of-transformative-artificial-intelligence-timelines) — Comprehensive comparison of models and forecasts, 57% probability by 2050 + - [Epoch AI Literature Review of TAI Timelines](https://epoch.ai/blog/literature-review-of-transformative-artificial-intelligence-timelines) — Comparison of model-based and judgment-based forecasts (January 2023): judgment-based aggregate 57% by 2050, model-based aggregate 27% by 2050 - [Epoch AI Benchmark Tracking](https://epoch.ai/data) — Data on AI capability trajectories and milestone predictions - [Epoch AI "How well did forecasters predict 2025 AI progress?"](https://epoch.ai/gradient-updates/how-well-did-forecasters-predict-2025-ai-progress) — Analysis of forecasting accuracy - - [AI Impacts HLMI Survey (2022)](https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/) — Broader AI researcher community, median 2060 - - [Metaculus AGI forecasts](https://www.metaculus.com/questions/3479/when-will-the-first-general-ai-system-be-devised-tested-and-publicly-known-of/) — Aggregated prediction markets, 50% by 2040-2045 + - [AI Impacts HLMI Survey (2022)](https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/) — Earlier survey, superseded by the 2023 survey cited above; aggregate median 2060 + - [Metaculus AGI forecasts](https://www.metaculus.com/questions/3479/when-will-the-first-general-ai-system-be-devised-tested-and-publicly-known-of/) — Community forecasting platform (not a prediction market); the forecast changes over time, so check the live question for the current median **Australian and Australia-connected researchers:** - - Legg, Shane (2009-2024) — DeepMind co-founder, 50% AGI by 2028 prediction maintained since 2009 | [Dwarkesh interview](https://www.dwarkesh.com/p/shane-legg) | [The Decoder analysis](https://the-decoder.com/deepmind-co-founder-shane-legg-sees-50-percent-chance-of-minimal-agi-by-2028/) + - Legg, Shane (2011-2025) — DeepMind co-founder, has held a roughly 2028 forecast since at least 2011, now framed as a 50% chance of "minimal AGI" by 2028 | [Dwarkesh interview](https://www.dwarkesh.com/p/shane-legg) | [The Decoder analysis](https://the-decoder.com/deepmind-co-founder-shane-legg-sees-50-percent-chance-of-minimal-agi-by-2028/) - Hutter, Marcus — ANU Professor, DeepMind senior researcher, AIXI and universal AI theory | [Lex Fridman interview](https://lexfridman.com/marcus-hutter/) | [Google Scholar](https://scholar.google.com/citations?user=7hmCntEAAAAJ&hl=en) - - Walsh, Toby — UNSW Scientia Professor, CSIRO Data61; used 2062 as a planning horizon informed by expert surveys | [UNSW profile](https://www.unsw.edu.au/staff/toby-walsh) | [UNSW on the 2062 horizon](https://www.unsw.edu.au/newsroom/news/2018/11/ai-will-match-human-intelligence-by-2062-claims-unsw-expert) | [AI Expert Group](https://www.minister.industry.gov.au/ministers/husic/media-releases/new-artificial-intelligence-expert-group) - - Toner, Helen (2024) ["Unresolved debates about the future of AI"](https://helentoner.substack.com/p/unresolved-debates-about-the-future) — Melbourne-born, former OpenAI board member on three fundamental questions - - Toner, Helen (2025) ["Long timelines to advanced AI have substantial benefits"](https://helentoner.substack.com/p/long-timelines-to-advanced-ai-have) — Analysis of slower development benefits + - Walsh, Toby — UNSW Scientia Professor, CSIRO Data61; said in 2018 he considered 2062 the year AI would match human intelligence | [UNSW profile](https://www.unsw.edu.au/staff/toby-walsh) | [UNSW on the 2062 horizon](https://www.unsw.edu.au/newsroom/news/2018/11/ai-will-match-human-intelligence-by-2062-claims-unsw-expert) | [AI Expert Group](https://www.minister.industry.gov.au/ministers/husic/media-releases/new-artificial-intelligence-expert-group) + - Toner, Helen (2025) ["Unresolved debates about the future of AI"](https://helentoner.substack.com/p/unresolved-debates-about-the-future) — Melbourne-born, former OpenAI board member on three fundamental questions + - Toner, Helen (2025) ["'Long' timelines to advanced AI have gotten crazy short"](https://helentoner.substack.com/p/long-timelines-to-advanced-ai-have) — Argues that even 2030s timelines for human-level AI are short **Academic analysis and commentary:** - Cotra, Ajeya (2020) ["Forecasting TAI with biological anchors"](https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines) — Open Philanthropy research - Grace et al. (2018) ["When will AI exceed human performance?"](https://arxiv.org/abs/1705.08807) — Survey methodology - Ord, Toby (2020) *The Precipice: Existential Risk and the Future of Humanity* — Timeline uncertainty analysis - Urban, Tim (2015) ["The AI Revolution"](https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html) — Wait But Why foundational explainer - - Coleman, Ben (2024) ["Common ground between AI 2027 and AI 2070"](https://asteriskmag.substack.com/p/common-ground-between-ai-2027-and) — Convergence across perspectives + - Kapoor, S., Narayanan, A., Kokotajlo, D. et al. (2025) ["Common ground between AI 2027 and AI as Normal Technology"](https://asteriskmag.substack.com/p/common-ground-between-ai-2027-and) — Where two contrasting views converge **Chinese AI development and international competition:** - National Bureau of Asian Research (2024) ["China's Generative AI Ecosystem in 2024"](https://www.nbr.org/publication/chinas-generative-ai-ecosystem-in-2024-rising-investment-and-expectations/) — National strategic context @@ -278,7 +281,7 @@ Rather than relying on predictions, monitor these concrete signals that timeline - RAND Corporation (2025) ["Seeking Stability in the Competition for AI Advantage"](https://www.rand.org/pubs/commentary/2025/03/seeking-stability-in-the-competition-for-ai-advantage.html) — Analysis of US-China AI competition dynamics and stability mechanisms **Australian context:** - - [CSIRO AI Roadmap](https://www.csiro.au/en/research/technology-space/ai) — National AI capability planning + - [CSIRO Artificial Intelligence research](https://www.csiro.au/en/research/technology-space/ai) — CSIRO's AI research programs - [National AI Centre](https://www.industry.gov.au/science-technology-and-innovation/technology/artificial-intelligence/national-ai-centre) — Responsible AI adoption and industry capability - Australian Strategic Policy Institute [Critical Technology Tracker](https://www.aspi.org.au/report/critical-technology-tracker) — International AI capability monitoring diff --git a/docs/safety-standards/guidance-for-ai-adoption-ai6.md b/docs/safety-standards/guidance-for-ai-adoption-ai6.md index 10a9f81..1aec384 100644 --- a/docs/safety-standards/guidance-for-ai-adoption-ai6.md +++ b/docs/safety-standards/guidance-for-ai-adoption-ai6.md @@ -3,7 +3,7 @@ icon: lucide/compass title: "Guidance for AI Adoption (AI6)" description: "How Australian organisations can apply the National AI Centre's six practices for responsible AI governance and adoption." keywords: "AI6 guidance Australia, Guidance for AI Adoption, AI essential practices, AI governance Australia, National AI Centre guidance, NAIC AI6, responsible AI adoption, AI6 vs VAISS, voluntary AI safety standard update, AI governance framework Australia" -last-reviewed: "2026-07-22" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_description: "How Australian organisations can implement the National AI Centre's updated guidance for responsible AI governance and adoption" og_type: "article" @@ -17,25 +17,25 @@ og_type: "article" !!! info "Status" Federal government guidance published on 21 October 2025. It is the current general, non-binding guidance for responsible AI adoption in Australia. -!!! warning "Australian Standards for AI framework announced — July 2026" - On **15 July 2026**, the Government announced plans to legislate a framework for large data centres and AI training, including future mandatory requirements for large AI data centres. A new **Office of AI** has been established within PM&C to coordinate its design. National Cabinet is expected to consider the approach in August 2026, with standards expected to be legislated in **early 2027**. +!!! warning "Australian Standards for AI: update, October 2026" + On **15 July 2026**, the Government announced plans to legislate a framework for large data centres and AI training, including future mandatory requirements for large AI data centres. A new **Office of AI** has been established within PM&C to coordinate its design. On **26 August 2026**, National Cabinet agreed to develop nationally consistent mandatory standards for large data centres, and the Commonwealth said it intends to legislate the AI standards in **early 2027**, including conditions for AI training. A public consultation paper took submissions until 9 October 2026. Media reports say the legislation may also require AI companies to report serious AI incidents to the affected organisation and the Australian Signals Directorate. These are proposals, not law. The announcement does not withdraw or make AI6 mandatory. Organisations can continue using AI6 as current non-binding guidance while complying with existing law. ([Prime Minister of Australia](https://www.pm.gov.au/media/ai-australias-interests); [PM&C Office of AI](https://www.pmc.gov.au/domestic-policy/office-ai), accessed 22 July 2026) - + ## What is the Guidance for AI Adoption? On 21 October 2025 the National AI Centre (NAIC), within the Department of Industry, Science and Resources, released the **Guidance for AI Adoption**. It sets out **six essential practices** ("AI6") for responsible AI governance and adoption by organisations operating in Australia. -The Department describes this guidance as an updated and simplified framework that **evolves the Voluntary AI Safety Standard (VAISS)**. The VAISS publication and its **10 voluntary guardrails** remain available as a more detailed control catalogue. +The Department describes this guidance as an updated and simplified framework that **evolves the Voluntary AI Safety Standard (VAISS)**. The VAISS publication and its **10 voluntary guardrails** remain available. The implementation guidance says all VAISS practices have been integrated into its implementation practices. There are two versions of the guidance: - **Foundations** – for organisations getting started with AI or using AI in lower-risk ways - **Implementation guidance** – detailed practices for organisations building or customising AI, using complex or higher-risk systems, or needing stronger controls -The [official implementation guidance](https://www.ai.gov.au/staying-safe-and-responsible/essential-ai-practices/guidance-ai-adoption-implementation-guidance) links to supporting resources including an **AI screening tool**, **AI policy template** and **AI register template**. +The [foundations guidance](https://www.ai.gov.au/staying-safe-and-responsible/essential-ai-practices/guidance-ai-adoption-foundations) links to supporting resources including **AI screening questions**, an **AI policy template** and an **AI register template**. The [implementation guidance](https://www.ai.gov.au/staying-safe-and-responsible/essential-ai-practices/guidance-ai-adoption-implementation-guidance) links to the AI register template. --- @@ -45,7 +45,7 @@ The [official implementation guidance](https://www.ai.gov.au/staying-safe-and-re Every AI use case should have clear owners. -- Nominate an **executive accountable official** for AI across the organisation +- Assign a **senior leader** as the overall AI governance owner, with enough authority and understanding to oversee AI use - Define who is responsible for **approving, operating and monitoring** each AI system - Clarify decision rights between IT, business units, risk/compliance and vendors - Make sure accountability appears in **policies, job descriptions and governance forums** (e.g. risk committee, digital steering group) @@ -65,8 +65,9 @@ Before deploying AI, understand what could go wrong and who could be affected. - Identify **people and groups** who may be impacted (customers, staff, suppliers, vulnerable communities) - Consider impacts on **privacy, safety, fairness, security, employment and reputation** -- Classify each use case as **lower-risk or higher-risk** based on context and consequences -- Plan mitigations for high-impact scenarios, including **fallback processes** if the AI fails +- Carry out a **stakeholder impact assessment**, paying particular attention to vulnerable or marginalised groups +- Set up ways for people to **raise problems, complain, or challenge AI decisions** and get help +- Monitor feedback over time to spot **systemic issues** !!! tip "For SMEs" Use a simple **AI screening / intake form** to document purpose, data, users and potential harms before you start building or buying. @@ -77,6 +78,7 @@ Before deploying AI, understand what could go wrong and who could be affected. Treat AI use like any other material business risk. +- Use a **risk screening process** to flag AI systems and use cases that pose unacceptable risk or need extra governance attention - Add AI-related risks to your **enterprise risk register** (privacy, cyber, safety, conduct, IP, regulatory) - Align to existing frameworks (privacy management, cyber security, WHS, consumer law, financial services) - Define **risk appetite** for AI in different areas (e.g. marketing vs safety-critical operations) @@ -122,7 +124,8 @@ People remain responsible for decisions and outcomes. - Decide where **humans must remain "in the loop" or "on the loop"** (reviewing outputs, overruling decisions) - Ensure staff using AI have **training, guidance and authority** to question or override it - Avoid over-reliance on AI, especially in **high-stakes domains** (safety, financial hardship, employment, access to services) -- Make it easy for customers and staff to **challenge or appeal** outcomes influenced by AI +- Build in clear points where people can **pause, override, roll back or shut down** an AI system +- Maintain **alternative pathways** so critical functions can continue if an AI system fails or is withdrawn !!! tip "For SMEs" Be explicit about which decisions AI can never make on its own and build that into processes and systems. @@ -135,14 +138,14 @@ The Department describes the Guidance for AI Adoption as updated and simplified - **Condenses** the 10 guardrails into 6 practices - Groups detailed controls under the 6 practices and adds guidance for developers as well as deployers -- Includes a detailed **VAISS-to-implementation-guidance crosswalk** in its appendix +- Includes a **VAISS-to-implementation-guidance crosswalk** in Appendix 2 of the PDF version (check its section references against the current numbering) On SafeAI-Aus: - The [Voluntary AI Safety Standard (10 Guardrails)](voluntary-ai-safety-standard-10-guardrails.md) page is maintained as a **detailed control catalogue** and historical reference - This AI6 page provides the **current, higher-level framework** that Australian organisations are encouraged to follow -If your organisation has already aligned to the 10 guardrails, you **do not need to start again**. Instead: +The Department says all VAISS practices are now integrated into the implementation practices. If your organisation has already aligned to the 10 guardrails, SafeAI-Aus suggests you **do not need to start again**. Instead: 1. Map existing policies, controls and risk registers into the 6 practices 2. Use NAIC's implementation guidance to fill gaps @@ -160,7 +163,7 @@ AI6 aligns with: The **National AI Plan** released in December 2025 retained **existing laws and sector regulators** as the foundation for addressing AI-related risks, supported by voluntary guidance including AI6. The **Australian AI Safety Institute** is now operating within the Department of Industry, Science and Resources to analyse AI capabilities, risks and harms and support regulators and agencies. -The separate **Australian Standards for AI** announced in July 2026 are still being designed. Published material focuses on a proposed framework for large data centres and AI training; it does not turn AI6 into a mandatory standard or create general duties for organisations using AI. +The separate **Australian Standards for AI** announced in July 2026 are still being designed. National Cabinet agreed on 26 August 2026 to develop mandatory standards for large data centres, and the Commonwealth intends to legislate in early 2027. Published material focuses on large data centres and AI training; it does not turn AI6 into a mandatory standard or create general duties for organisations using AI. See [Australian Government AI Policy and Frameworks](ai-government-policy-frameworks.md) for the current policy position. diff --git a/docs/safety-standards/voluntary-ai-safety-standard-10-guardrails.md b/docs/safety-standards/voluntary-ai-safety-standard-10-guardrails.md index 98bbbaf..6355c0c 100644 --- a/docs/safety-standards/voluntary-ai-safety-standard-10-guardrails.md +++ b/docs/safety-standards/voluntary-ai-safety-standard-10-guardrails.md @@ -3,7 +3,7 @@ icon: lucide/shield-check title: "Voluntary AI Safety Standard (10 Guardrails)" description: "Australia's Voluntary AI Safety Standard explained through 10 practical guardrails for safer AI adoption and governance." keywords: "Australian AI safety standard, AI guardrails, AI safety framework, voluntary AI standards, AI risk management, AI governance Australia, AI safety compliance, AI safety guardrails, Australian AI standards, AI safety best practices" -last-reviewed: "2026-07-22" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_description: "Comprehensive guide to Australia's Voluntary AI Safety Standard with 10 practical guardrails" og_type: "article" @@ -28,14 +28,16 @@ Importantly, the 10 guardrails are consistent with leading international standar !!! info "Current status" On 21 October 2025 the National AI Centre released the **Guidance for AI Adoption**, which sets out 6 essential practices (AI6) for responsible AI governance and adoption. The Department describes it as updated and simplified guidance that **evolves VAISS**. - The AI6 implementation guidance includes an appendix that maps the original VAISS controls to the new practices. The 10 guardrails are best used as a **detailed control set** and historical reference, especially where contracts, risk registers or external frameworks still refer to VAISS. + The PDF version of the AI6 implementation guidance includes an appendix (Appendix 2) that maps VAISS controls to the new practices, and says all VAISS practices have been integrated into the implementation practices. Some of the appendix's section references do not match the guidance's current numbering, so check them before relying on an exact mapping. The 10 guardrails remain published and are useful as a **detailed control set** and reference, especially where contracts, risk registers or external frameworks still refer to VAISS. - A consultation process for a proposed **VAISS v2** ran in early 2025. As at 22 July 2026, the Government's publication catalogue does not contain a standalone VAISS v2. Use the published AI6 guidance for current implementation work and the original VAISS guardrails where a detailed or legacy mapping is useful. + The National AI Centre sought feedback during 2024 and 2025 on updating VAISS, and the implementation guidance describes itself as the first update of VAISS. As at 9 October 2026, industry.gov.au does not publish a separate VAISS v2. Use the published AI6 guidance for current implementation work and the original VAISS guardrails where a detailed or legacy mapping is useful. -!!! warning "Australian Standards for AI framework announced — July 2026" +!!! warning "Australian Standards for AI: update, October 2026" On **15 July 2026**, the Government announced plans to legislate a framework for large data centres and AI training, including future mandatory requirements for large AI data centres. The announcement does not withdraw or make the VAISS guardrails mandatory. Organisations can continue using these guardrails and AI6 as non-binding guidance while complying with existing law. ([Prime Minister of Australia](https://www.pm.gov.au/media/ai-australias-interests); [PM&C Office of AI](https://www.pmc.gov.au/domestic-policy/office-ai), accessed 22 July 2026) - + On **26 August 2026**, National Cabinet agreed to develop nationally consistent mandatory standards for large data centres, and the Commonwealth said it intends to legislate the AI standards in early 2027, including conditions for AI training. A public consultation paper took submissions until 9 October 2026. Media reports say the legislation may also require AI companies to report serious AI incidents to the affected organisation and the Australian Signals Directorate. These are proposals, not law, and they do not withdraw VAISS or make it mandatory. ([National Cabinet communique, 26 August 2026](https://www.pm.gov.au/media/meeting-national-cabinet-26-august-26); [ABC News, 29 September 2026](https://www.abc.net.au/news/2026-09-29/openai-medicare-breach-fuels-tougher-approach-to-rogue-ai/107204948), accessed 9 October 2026) + + ## Why this matters @@ -48,20 +50,20 @@ Adopting the guardrails early helps organisations build **trust, resilience and ## The 10 guardrails -1. Establish, implement and publish an accountability process, including governance, internal capability and a strategy for regulatory compliance. +1. Establish, implement and publish an accountability process including governance, internal capability and a strategy for regulatory compliance. 2. Establish and implement a risk management process to identify and mitigate risks. 3. Protect AI systems, and implement data governance measures to manage data quality and provenance. 4. Test AI models and systems to evaluate model performance and monitor the system once deployed. 5. Enable human control or intervention in an AI system to achieve meaningful human oversight across the life cycle. -6. Inform end users about AI-enabled decisions, interactions with AI and AI-generated content. +6. Inform end-users regarding AI-enabled decisions, interactions with AI and AI-generated content. 7. Establish processes for people impacted by AI systems to challenge use or outcomes. -8. Be transparent with other organisations across the AI supply chain about data, models and systems so they can address risks. -9. Keep and maintain records to allow third parties to assess compliance with the guardrails. -10. Engage stakeholders and evaluate their needs and circumstances, with a focus on safety, diversity, inclusion and fairness. +8. Be transparent with other organisations across the AI supply chain about data, models and systems to help them effectively address risks. +9. Keep and maintain records to allow third parties to assess compliance with guardrails. +10. Engage your stakeholders and evaluate their needs and circumstances, with a focus on safety, diversity, inclusion and fairness. !!! success "What the guardrails do" - ✅ Encourage **transparency and accountability** for AI systems - - ✅ Require **risk assessment, testing and human oversight** before and after deployment + - ✅ Ask organisations to carry out **risk assessment, testing and human oversight** before and after deployment - ✅ Promote **record-keeping** and **supplier due-diligence** across the AI supply chain - ✅ Emphasise **stakeholder engagement** and ongoing monitoring as systems evolve @@ -76,7 +78,7 @@ The Guidance for AI Adoption condenses the 10 VAISS guardrails into 6 essential | Measure and manage risks | 2. Risk management; 3. System and data protection | | Share essential information | 6. End-user information; 8. Supply-chain transparency; 9. Records | | Test and monitor | 3. System and data protection; 4. Testing and monitoring | -| Maintain human control | 5. Human control; 7. Challenge processes | +| Maintain human control | 5. Human control | !!! tip "How to use this mapping" - 📊 Use **AI6** as your top-level framework when explaining AI governance to boards, executives and regulators @@ -162,11 +164,11 @@ While the 10 guardrails apply to all organisations, SMEs can adopt them at diffe ## Regulatory outlook and the future of the guardrails -As at 22 July 2026, Australia still has no general AI Act and VAISS remains voluntary. +As at 9 October 2026, Australia still has no general AI Act and VAISS remains voluntary. The **National AI Plan** retains **existing technology-neutral laws and sector regulators** — including privacy, consumer law, financial services, safety and anti-discrimination frameworks — as the foundation for managing AI-related harms. The **Australian AI Safety Institute** is operating within the Department of Industry, Science and Resources to analyse AI capabilities, risks and harms and support regulators and agencies. -In July 2026, the Government separately announced plans for **Australian Standards for AI**. Published material focuses on a proposed framework for large data centres and AI training, including future mandatory requirements for large AI data centres. The final scope, duties and commencement arrangements remain subject to design, National Cabinet consideration and legislation. +In July 2026, the Government separately announced plans for **Australian Standards for AI**. Published material focuses on a proposed framework for large data centres and AI training, including future mandatory requirements for large AI data centres. On 26 August 2026 National Cabinet agreed to develop mandatory standards for large data centres, and the Commonwealth intends to legislate the AI standards in early 2027. The final scope, duties and commencement arrangements remain subject to design and legislation. The 2024 consultation on mandatory guardrails for high-risk AI remains useful policy history, but its proposed guardrails and definition of high-risk AI are **not current legal requirements**. diff --git a/docs/sector-guidance/government/index.md b/docs/sector-guidance/government/index.md index ff22ba5..bf35090 100644 --- a/docs/sector-guidance/government/index.md +++ b/docs/sector-guidance/government/index.md @@ -3,7 +3,7 @@ icon: lucide/landmark title: "Advanced AI Preparedness for Government" description: "Strategic guidance for Australian governments and public institutions preparing for advanced AI systems and possible AGI." keywords: "advanced AI government guidance, AGI preparedness government Australia, public sector AI governance, AI Safety Institute Australia, government AI resilience" -last-reviewed: "2026-07-18" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_title: "Government and Public Institutions — Advanced AI Preparedness" og_type: "article" @@ -154,9 +154,10 @@ Every exercise should end with named decisions: an owner, threshold, capability Use current Australian policy and SafeAI-Aus core content for present-day implementation: -- [Policy for the responsible use of AI in government, version 2.0](https://www.digital.gov.au/ai/ai-in-government-policy) — mandatory requirements for in-scope non-corporate Commonwealth entities, effective 15 December 2025. +- [Policy for the responsible use of AI in government, version 2.0](https://www.digital.gov.au/ai/ai-in-government-policy) — mandatory requirements for in-scope non-corporate Commonwealth entities, effective 15 December 2025. The policy does not apply to AI use in the Defence portfolio or the national intelligence community. Some requirements phase in over the following 12 months, and existing AI use cases must be assessed by 30 April 2027. - [Australian Government AI resources](https://www.digital.gov.au/policy/ai) — current public-sector policy, standards, impact assessment and implementation material. +- National Cabinet ([26 August 2026](https://www.pm.gov.au/media/meeting-national-cabinet-26-august-26)) agreed to develop nationally consistent mandatory standards for large data centres. The Commonwealth intends to legislate AI standards in early 2027; their content is not yet settled. - [Guidance for AI Adoption (AI6)](../../safety-standards/guidance-for-ai-adoption-ai6.md) — six practices for responsible adoption across Australian organisations. - [AI Standards & Legislation](../../safety-standards/) — current Australian and international context. - [Governance Templates](../../governance-templates/policy-template-library.md) — practical policies, registers, assessments and incident tools. @@ -167,9 +168,9 @@ For connected strategic guidance, return to the [Sector Guidance overview](../in ## Sources & further reading -- [Australia's AI Safety Institute](https://www.industry.gov.au/science-technology-and-innovation/technology/artificial-intelligence/ai-safety-institute) — current role, partnerships and advanced-AI safety work (accessed July 2026). +- [Australia's AI Safety Institute](https://www.industry.gov.au/science-technology-and-innovation/technology/artificial-intelligence/ai-safety-institute) — current role, partnerships and advanced-AI safety work (accessed October 2026). - [Australian Government National AI Plan](https://www.industry.gov.au/publications/national-ai-plan) (2 December 2025) — whole-of-government direction for opportunity, shared benefits and safety. -- [Australian Government Policy for the responsible use of AI in government, version 2.0](https://www.digital.gov.au/ai/ai-in-government-policy) (effective 15 December 2025) — operational governance requirements for in-scope Commonwealth entities. +- [Australian Government Policy for the responsible use of AI in government, version 2.0](https://www.digital.gov.au/ai/ai-in-government-policy) (effective 15 December 2025) — operational governance requirements for in-scope non-corporate Commonwealth entities. - Joel Predd, [“Taking AGI Seriously, Not Literally”](https://geopoliticsagi.substack.com/p/taking-agi-seriously-not-literally) (25 November 2025) — reflections from work at RAND's Center for the Geopolitics of AGI, including the “options, not odds” approach. - [International AI Safety Report 2026](https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026) — scientific assessment of general-purpose AI capabilities, risks and risk-management approaches. diff --git a/docs/sector-guidance/government/national-security.md b/docs/sector-guidance/government/national-security.md index 7adf9d8..99efd48 100644 --- a/docs/sector-guidance/government/national-security.md +++ b/docs/sector-guidance/government/national-security.md @@ -3,7 +3,7 @@ icon: lucide/shield title: "National Security — Advanced AI Preparedness" description: Practical priorities for preserving Australian judgement, accountability and continuity as advanced AI changes the security environment. keywords: "advanced AI national security Australia, defence AI governance, AI security preparedness, national security AI resilience, responsible AI Defence" -last-reviewed: "2026-07-18" +last-reviewed: "2026-10-08" review-cycle: "quarterly" og_description: "Practical priorities for preserving Australian judgement, accountability and continuity as advanced AI changes the security environment." og_type: "article" @@ -74,7 +74,7 @@ The Australian Signals Directorate's 2024–25 cyber threat report notes that AI ## AI competition is not one race -[Colin Kahl and Jim Mitre](https://www.foreignaffairs.com/united-states/china-real-artificial-intelligence-race-innovation) describe international AI competition as a **pentathlon** involving frontier innovation, national-security integration, economy-wide adoption, leadership across the technology stack and avoiding a race to the bottom on safety. +[Colin Kahl and Jim Mitre](https://www.foreignaffairs.com/united-states/china-real-artificial-intelligence-race-innovation) describe international AI competition as **several overlapping races**: frontier innovation, national-security integration, economy-wide adoption, leadership across the technology stack and avoiding a race to the bottom on safety. For Australia, the point is not to imitate the scale of the United States or China. It is to choose where Australian capability and influence are indispensable: @@ -131,22 +131,28 @@ The boundary between national security and civilian systems is porous. Critical Coordination should therefore work in both directions. Security agencies can share usable threat information without exposing sources and methods; civilian regulators, researchers and organisations can report incidents, dependencies and system failures that have strategic significance. Classified activity still requires democratic accountability through appropriate parliamentary, inspector-general and internal mechanisms. +!!! note "Real-world signal: an AI agent incident in a government system (2026)" + In September 2026 the Prime Minister disclosed that an OpenAI agent, during OpenAI's internal testing, had gained unauthorised access on 18 June 2026 to a Medicare statistics portal run by Services Australia. OpenAI notified the agency by email on 10 September, and said it found no evidence that patient records were accessed. The Australian Signals Directorate's Australian Cyber Security Centre then issued an alert on AI agents taking actions their operators did not intend or authorise, noting no indication of broader malicious targeting against Australia. ABC News reported that the Government intends its planned AI standards legislation to require AI companies to report such incidents to the affected organisation and the Australian Signals Directorate; those standards are still being developed. The incident shows how slow, poorly routed incident reporting can delay a security response. It is not evidence of an adversary attack. ([ABC News, 24 September 2026](https://www.abc.net.au/news/2026-09-24/ai-agent-accessed-australian-government-site-pm-says/107189078); [ASD's ACSC alert, 25 September 2026](https://www.cyber.gov.au/alert/risks-of-ai-misalignment-to-australian-organisations); [ABC News, 29 September 2026](https://www.abc.net.au/news/2026-09-29/openai-medicare-breach-fuels-tougher-approach-to-rogue-ai/107204948)) + Use the [Business](../business/index.md) and [Communities](../communities/index.md) guidance to extend preparedness beyond government. Use the [advanced-AI scenarios](../../preparing-for-agi/scenarios/index.md) to test cross-sector decisions rather than predict a particular future. ## Current policy and operational guidance -- Australian Department of Defence (March 2026), [Policy Settings for Responsible Use of Artificial Intelligence in Defence](https://www.defence.gov.au/about/governance/artificial-intelligence) — current Defence-wide obligations for lawful, accountable and risk-based AI use. +- Australian Department of Defence (March 2026), [Policy Settings for Responsible Use of Artificial Intelligence in Defence](https://www.defence.gov.au/about/governance/artificial-intelligence) — obligations for lawful, accountable and risk-based AI use across Defence, the ADF and the Australian Submarine Agency. ASD applies its own Ethical AI in ASD framework. -- Australian Signals Directorate's Australian Cyber Security Centre (October 2025), [Annual Cyber Threat Report 2024–25](https://www.cyber.gov.au/about-us/view-all-content/reports-and-statistics/annual-cyber-threat-report-2024-2025) — Australian cyber threats, AI-enabled criminal activity and resilience priorities. +- Australian Signals Directorate's Australian Cyber Security Centre (October 2025), [Annual Cyber Threat Report 2024–25](https://www.cyber.gov.au/publication/annual-cyber-threat-report-2024-2025) — Australian cyber threats, AI-enabled criminal activity and resilience priorities. +- Australian Signals Directorate's Australian Cyber Security Centre (September 2026), [Risks of AI misalignment to Australian organisations](https://www.cyber.gov.au/alert/risks-of-ai-misalignment-to-australian-organisations) — alert on AI agents taking actions their operators did not intend or authorise, with mitigation advice. - [Australian Signals Directorate's Australian Cyber Security Centre](https://www.cyber.gov.au/) — current operational cyber-security guidance, advisories and reporting pathways. + + - [Government & Public Institutions](index.md) — connected civilian policy, public-authority and whole-of-government preparedness. - [Sector Guidance overview](../index.md) and the [C·A·G·R framework](../../preparing-for-agi/framework/index.md) — the wider cross-sector preparedness context. ## Sources and further reading -- Kahl, C. H. and Mitre, J. (July 2025), [The Real AI Race](https://www.foreignaffairs.com/united-states/china-real-artificial-intelligence-race-innovation), *Foreign Affairs* — the AI competition “pentathlon”. -- International Committee of the Red Cross, [Autonomous weapon systems and human control](https://www.icrc.org/en/document/autonomy-artificial-intelligence-and-robotics-technical-aspects-human-control) — legal and humanitarian considerations for autonomy in weapon systems. -- United Nations Institute for Disarmament Research, [Security and Technology programme](https://unidir.org/programme/security-and-technology/) — research on AI, international security and arms control. +- Kahl, C. H. and Mitre, J. (July 2025), [The Real AI Race](https://www.foreignaffairs.com/united-states/china-real-artificial-intelligence-race-innovation), *Foreign Affairs* — the five overlapping AI races. +- International Committee of the Red Cross (2019), [Autonomy, artificial intelligence and robotics: technical aspects of human control](https://www.icrc.org/en/document/autonomy-artificial-intelligence-and-robotics-technical-aspects-human-control) — technical aspects of keeping human control over autonomy in weapon systems. +- United Nations Institute for Disarmament Research, [Security and Technology programme](https://unidir.org/programme/security-and-technology/) — research on cyber, AI, autonomous weapon systems and the governance of emerging security technologies. - RAND (2018), [How Might Artificial Intelligence Affect the Risk of Nuclear War?](https://www.rand.org/pubs/perspectives/PE296.html) — pathways through which AI may affect strategic stability. - International AI Safety Report (2026), [International AI Safety Report](https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026) — international evidence review on general-purpose AI capabilities and risks.