Rishi Sunak pushes countries to label AI as capable of causing ‘catastrophic harm’

Britain wants countries to sign up to a joint position that outlines particular concerns for AI’s impact on cybersecurity and biotechnology, according to a draft communique circulated to attendees and seen by Bloomberg. Officials aim to hammer out final wording of the communique by Oct. 25, a separate document showed.

“There is potential for significant, even catastrophic, harm, either deliberate or unintentional, stemming from the most dangerous capabilities of these AI models,” according to the draft, dated Oct. 16. 

The draft also highlights the “transformative opportunities” of AI, particularly in public services like health, education, science and clean energy. This will form a prominent part of the summit, according to a person familiar with the matter.

The meeting at Bletchley Park, home to British code breakers including Alan Turing during World War II, is part of an initiative by Sunak to position the UK at the vanguard of AI and help inform regulatory approaches and establish “guardrails” for the industry around the world. Some 100 people from 28 nations are expected to attend, including business executives, industry experts and global leaders.

The document was still subject to change, although Sunak’s administration is close to producing a final version, said the person, who asked not to be named while discussing information that hasn’t yet been publicly released.  

Global leaders are also set to call for increased transparency by companies developing AI technologies, and will focus on creating “clear evaluation metrics, tools for safety testing, and developing relevant public sector capability and scientific research,” according to the document.

“Particular safety risks arise at the ‘frontier’ of general-purpose AI, as well as in some cases specific narrow AI that could exhibit dangerous capabilities,” the draft says. “The most significant of these risks arise from potential intentional misuse or issues of control, where AI systems may seek to increase their own influence and reduce human control, and these issues are in part because those capabilities are not fully understood.”

The draft and other documents circulated among participants also say:

• The European Commission is pushing for a process for international collaboration on frontier AI safety.
• The bloc is also keen for any agreed approach to fall in line with its own legislation on AI, including a focus on its potential misuse in cyber-attacks and the ability for advanced systems to lose control.
• Developers of “unusually powerful and potentially dangerous frontier AI capabilities have a particularly strong responsibility for ensuring the safety of these capabilities.”
• All “relevant actors” are urged to “provide context-appropriate transparency and accountability on their plans to measure, monitor and mitigate potentially dangerous capabilities and the associated effects that may emerge.”

The UK is aiming to have follow-up meetings on progress toward managing the opportunities and risks of AI every six months following the summit, according to another person familiar with the discussions. 

The UK has also proposed assembling an expert panel of academics from different countries to compile an annual “state of the science” report on AI safety, according to one of the documents accompanying the draft. The annual report would summarize existing research rather than producing new material. Under the plan, countries attending the summit will nominate their own experts to join the panel.

“For the common good, AI must be designed, developed, deployed, and used in such a way as to be human-centric, safe, trustworthy and responsible,” the draft communique says.