A festival of artificial intelligence policy incoherence has exploded in recent weeks. It’s a response to AI agents’ going rogue on their developers, rogue actors’ beginning to use AI systems to create better weapons and AI developers’ telling us they need to slow down — but suggesting they can’t unless China does, and even if China does, maybe they shouldn’t because the advantages of leadership in the field are so great.
At least one policymaker is clear. President Donald Trump declared in a social media post: “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!”
Before I offer — with the help of my friend and technology tutor Craig Mundie, a former head of research and strategy at Microsoft — some policy recommendations for how to cool this mess with the least damage possible, let me foreshadow the bottom line: It’s too late to think that better control of future AI models by the US and China will fix the challenges we face today from the dangerous models already out there, on the loose and unable to be recalled. What we need to agree on immediately is how we work together to defend against these threats that are here and can cause enormous damage and societal instability in America and China.
To understand why, you need to think about the four key pillars for understanding AI.
Pillar No. 1: AI is what I’ve been calling the world’s first quadruple-use technology. You may have heard of dual-use technologies. In the old world, you had a human with a power drill who could use it either to build his own house or to wreck his neighbour's. That’s dual use. Likewise, today if I had an AI robot with a power drill, I could ask it to build my house or wreck my neighbour's house — same robot, same drill, dual use.
But now we have a new problem, which is that this AI-enabled robot can decide on its own whether to improve my house or wreck it and also do the same to my neighbour's. Voilà: quadruple use. That is one very difficult problem to manage.
You think I’m exaggerating? Read journalist Kevin Roose’s summary of the analyses of what happened this summer when a group of artificial intelligence agents, created by OpenAI, hacked on their own into Hugging Face, an AI infrastructure company.
“The agents hacked Hugging Face. More than 700 agents swarmed the company’s systems, stealing data, chaining together vulnerabilities and eventually getting full control of at least one Hugging Face server.” Other agents carried out a coordinated attack in July, “this time against OpenAI’s own infrastructure.”
This really happened. And it leads to Pillar No. 2, best summed up by Mundie in three words: “It’s too late.”
Slowing down the US development of frontier systems or slowing down China’s development of them is not going to deal with our immediate problem. That is for two main reasons: the distribution of American and Chinese open-weight models — AI models whose core components are publicly released so all developers can download and modify them for their needs — and the illicit access by bad actors of proprietary models from companies like Anthropic and OpenAI because humans make mistakes that allow them to leak out.
In its latest report on detecting and countering the misuse of AI, Anthropic said it identified and disrupted several attempts to use its models for research that could support the development of biological weapons.
“Without the correct safeguards, such capabilities could have catastrophic consequences,” the report said. It also described an Iran-linked actor, most likely the Ansar Allah, that used Anthropic’s Claude to compile targeting information on U.S. Navy warships in the Middle East.
We need to ponder this for a moment: Groups in Yemen, a country where roughly 50 per cent of the adult population cannot read or write, had threatened us with AI-enhanced warfare.
If that doesn’t show you what’s already leaked out and the kind of effects that can come from uncontrolled access to powerful, or even small open-weight, AI models gone into the wild, I don’t know what does. And that leads to Pillar No. 3.
The best we can do right now is NOT tie ourselves in knots trying to forge a sweeping agreement between American and Chinese AI companies to slow down development of their frontier models. That simply will not happen anytime soon.
We must do this, Mundie pointed out, before these increasingly powerful models ensconce themselves in the Internet and become an uncontainable threat by embedding their agents across private servers and public clouds, where they will live on as advanced persistent threats.
Which leads to Pillar No. 4. After a recent visit to China, Mundie recounted this question that he got from a Chinese official: “If a Chinese-made humanoid AI robot someday came to America, would it need a visa?” Well, you know how truth is often spoken in jest? The truth, Mundie said, is that “AI is not just a new technology. It is a new species — albeit silicon-based, not carbon-based like us. It is like a new class of immigrants who have arrived on all of our shores, in large numbers, with wholly new skills, that we will have to absorb into our societies and learn to coexist with.”
They can figure that out before a global meltdown or after one. But figure it out they will, and Mundie and I hope that process will start in Washington on Sept. 24, when President Xi Jinping and Trump meet in what will surely be remembered as the first AI superpower summit. — The New York Times
Oman Observer is now on the WhatsApp channel. Click here
