There is a lot of talk in Beijing this week over when President Donald Trump and President Xi Jinping of China will meet face to face. Some Chinese experts say the two leaders need to wait a few months until Trump decides exactly what tariffs he is going to impose on China — and sees what China will do in response.
Can I just butt in and say: “Excuse me, Mr Presidents, but you two need to get together, like, tomorrow. But it’s not to discuss the golden oldies — tariffs, trade and Taiwan.
“There is an earthshaking event coming — the birth of artificial general intelligence. The United States and China are the two superpowers closing in on AGI — systems that will be as smart or smarter than the smartest human; and able to learn and act on their own. Whatever you both may think you’ll be judged on by history, I assure you that whether you collaborate to create a global architecture of trust and governance over these emerging super intelligent computers, so humanity gets the best out of them and cushions their worst, will be at the top.” I realise many will consider this wasted breath with all the turmoil unleashed by the new administration in Washington, but that will not deter me from making the point as loudly as I can. Because what Soviet-American nuclear arms control was to world stability since the 1970s, US-Chinese AI collaboration to make sure we effectively control these rapidly advancing AI systems will be for the stability of tomorrow’s world.
AI systems and humanoid robots offer so much potential benefit to humanity, but they could be hugely destructive and destabilising if not embedded with the right values and controls. In addition, this new age must be defined by a lot of planning about what humans will do for work and how to preserve the dignity they derive from work, when machines will be able to do so many things better than people. Millions of people possibly losing their jobs and dignity at the same time is a prescription for disorder.
New York Times technology writer Kevin Roose recently observed that full-on AGI is coming faster than most anyone thought — “very soon — probably in 2026 or 2027 but possibly as soon as this year.” AGI is the holy grail of AI — single systems that can master math, physics, biology, chemistry, material science, Shakespeare, poetry and literature as well as the smartest humans but that can also reason across all of them and see connections no human polymath ever could.
As Craig Mundie, a former chief research and strategy officer for Microsoft, put it to me: Probably before the end of Trump’s presidency, we will have not just birthed a new computer tool; “we will have birthed a new species — the super intelligent machine.” “Our species is carbon-based. This new one is silicon-based,” Mundie explained. “Therefore, we need to immediately begin to chart a path to coexist with this new super intelligent species and ultimately coevolve with it.”
Before these AGI systems take hold and scale up, we need the two superpowers to get serious about devising a regulatory and technological framework that ensures an agreement for imbuing these systems with some kind of moral reasoning and embedded usage controls so they are prevented from being used by rogue actors for globally destabilising activities or going rogue themselves. We need a system of governance that ensures that AI systems always operate and police themselves in alignment with both human and machine well-being.
Once AGI arrives, if we are not assured that these systems will be embedded with common trust standards, the United States and China will not be able to do anything together. Neither side will trust anything it exports or imports to the other, because AI will be in everything that is digital and connected. That is your car, your watch, your toaster, your favorite chair, your implant, your notepad. So if there is no trust between us and China; and each of us has our own AI systems, it will be the TikTok problem on steroids. A lot of trade will just grind to a halt. We’ll just be able to sell each other soybeans for soy sauce. It will be a world of high-tech feudalism.
I was taken with how Israeli historian Yuval Noah Harari, who addressed a packed audience of mostly Chinese people at the forum’s session on AI, put it.
“We should build more trust between humans before we develop truly super intelligent AI agents,” Harari said. “But we are now doing exactly the opposite. All over the world, trust between humans is collapsing. Too many countries think that to be strong is to trust no one and be completely separated from others. If we forget our shared human legacies and lose trust with everyone outside us, that will leave us easy prey for an out-of-control AI.” Together humans can control AI, he added, “but if we fight one another, AI will control us.” In this specific endeavour of creating trusted AI, I don’t hesitate to say I wish Xi and Trump much success — and fast. — The New York Times
Oman Observer is now on the WhatsApp channel. Click here
