The question isn’t whether we should slow down AI development. It’s whether we still can

Open this photo in gallery:

We are all going to die. Well, of course: None of us is going to live forever. Death waits for everyone. It is the prospect of all of us dying at the same time – the death of humanity itself – that tends to concentrate the collective mind.

It says a lot about the existential crisis we are now in that there are not one but several possible scenarios for human extinction currently vying for our attention. There is that old favourite, a third World War, given fresh impetus by Russia’s mounting attacks on Europe, whose purpose seems to be to test NATO’s resolve, at a moment of maximum uncertainty and division in the alliance. Already Europe’s citizens are being advised to stock up on canned goods, dig shelters and otherwise prepare for war.

Of course, Armageddon may not arrive in the form of a Russian assault on Europe. China may invade Taiwan first, again capitalizing on the democratic world’s divisions, which is a gentler way of saying “the presence of a madman in the White House.” Or the two may time their attacks to coincide.

But why leap to worst-case scenarios? Quite possibly, in the middle of this unprecedented global security crisis, as the attacks and counterattacks multiply in rapid succession, strategists on both (all three?) sides will nevertheless be able to prevent the final leap into the abyss.

Then there is the prospect, or rather near-certainty, of another pandemic. As with nuclear war, we should resist assuming this implies the death of literally every human being on Earth. The more deadly the disease, as we all learned during the COVID-19 pandemic, the harder it is for the virus to find new transmission agents. COVID-19 itself killed a mere 15 to 18 million worldwide in its first two years. Even the Black Death only killed an estimated 30 to 60 per cent of the European population.

Canada signs international statement on AI safety, joining calls for stronger oversight

Still, it’s possible that a virus could emerge that had such a long asymptomatic phase that it could spread rapidly before killing its hosts, or indeed before they were even aware they had the disease. Or one that could live for longer outside the human body than those with which we are familiar, or that was transmitted back and forth between humans and animals. So there’s that.

And then there is AI.

It seems more than a little strange that it took the resignation of a relatively junior researcher at Anthropic, one of the leading AI firms (along with OpenAI and Google DeepMind), to interest the media and the public in the possibility that AI might, as the phrase has it, “kill us all.”

Until now the discussion of AI’s possible ills had been largely confined to whether it would wipe out all human employment, destroy our capacity to reason, empower the surveillance state on a scale hitherto unimaginable, and (once it is widely understood what a house of cards the AI companies’ finances are) set off a worldwide capital-market collapse.

And yet some of the leading figures in AI have been telling us for years of their concern that the technology was advancing so fast that it could soon spin out of their control – indeed, that it could become so intelligent, so vastly more intelligent than we are, that it would control us, if it bothered with us at all.

AI leaders brief UN as warnings over technology’s ‘real and imminent’ dangers grow

It didn’t help that these warnings of where they were headed tended to be bundled with boasts of how far they had advanced already: We believe there is a 10-per-cent chance that this technology will destroy all human life, and by the way, we are releasing version 5.0 today.

It sounded too abstract to pay it too much attention (what does a 10-per-cent probability mean, anyway? Is it going to rain frogs and locusts or isn’t it?) and the cognitive dissonance (if it’s such a threat to humanity, why are you still building it?) made it easy to squirrel away somewhere at the back of the brain along with the other threats to our existence.

Open this photo in gallery:

But then there was the break-in at Hugging Face, the AI developers’ platform, in which hundreds of AI agents spawned by OpenAI, having earlier escaped the “sandbox” in which they were supposedly contained, conspired to hack into the site’s servers, in search of material with which to cover up their earlier attempts to cheat on a test OpenAI had set for them.

There followed reports of similar exploits by agents at Anthropic, as well as attempts by various “bad actors” to turn Anthropic’s technology to lethal ends. The drumbeat of revelations contributed to a rising sense of unease: How many more such incidents had there been, that we had not been told about? Were the companies even aware themselves?

So the atmosphere was already volatile: All it took was the spark of a viral resignation post to set it aflame, not least because senior officials at both companies, rather than deny the claims, seemed to agree with them. Still, it remained hard to take on board. It all seemed so unlikely, so preposterous, if only because of the extremity of the threat: The death of all human life? Really?

Australia says OpenAI agent hacked government website, accessed files

And so, inevitably, the deflections, red herrings and conspiracy theories began to pour forth. It was all a psy-op, ran one theory, an attempt to escape legal liability. When executives at both OpenAI and Anthropic called for regulations to slow the pace of development at the technology’s cutting-edge, it was dismissed as a classic attempt at “regulatory capture,” designed to lock in the big firms’ advantage over smaller competitors. The term “moral panic,” long favoured by the sophisticated and the world-weary, was regularly invoked.

Simple solutions followed. Why not just unplug the machines, before they could do much harm? It’s illegal to make a hazardous product, ran another line: Why not just punish the companies if they do? The problem wasn’t rogue AI agents, it was explained, so much as it was lax controls. Much discussion was devoted to denouncing language that tended to “anthropomorphize” the agents, or that implied they acted with conscious ill intent. Skeptics demanded explanations of precisely how the AI was going to kill us all.

Yet another line of argument concerned itself with explaining why, even if AI did pose an existential threat, nothing could be done, or at least why nothing would be done. How could the United States regulate, so long as China did not? How could any company pause its research, if its competitors refused? By the time Donald Trump weighed in with his opinion that it was all a “hoax,” like climate change or “Russia Russia Russia,” it was baked in. Only Democrats and other congenital bed-wetters could possibly take this seriously.

It all has a very March, 2020, vibe: March 10, to be specific, the day before Tom Hanks announced he had COVID-19, when it was still possible to believe that COVID-19 was something that happened in other places, far away, or so far as it was happening here would grow at the kind of manageable or at least predictable or at any rate comprehensible rate with which we were familiar. It wasn’t just that we did not expect the kind of exponential growth in cases that was soon to inundate us. It hadn’t even occurred to us.

Doomsday warnings over AI are growing louder. What should we make of them?

Something like the same failure of the imagination seems to be at work now. If it helps, put aside whether AI is likely to literally kill every human being on Earth. It is surely sufficient if it kills even a substantial portion of us, no? If only one in five of the 8 billion or so of us die that’s still 1.6 billion lives. Would that be an acceptable outcome?

Understand, too, where the “kill us all” projections come from. It isn’t that AI would set out to systematically exterminate us, one by one. It is that it would take actions, in pursuit of its own objectives, that had the incidental effect of making human life impossible – in the same way as humans might act, and have, in such a way as to destroy another species.

Open this photo in gallery:

This is where we have to think in exponential terms. As rapidly as AI has progressed in recent years – and it has progressed much faster than anyone had predicted – it is on the cusp of much faster progress still: incomprehensibly faster. Rather than human coders, the AI companies are increasingly giving the job of upgrading their apps to the apps themselves, assigning them to edit their own code. With better code, the apps grow more intelligent, thus able to write still better code, making them still more intelligent, and so on: a process known as recursive self-improvement.

It’s easy to see how this makes nonsense of “just pull the plug” solutions, particularly once a machine has been plugged into the internet. By the time their human minders realize there is a problem, an AI might have replicated itself any number of times on any number of other machines.

For all we know they may already have. Examples of AI models deceiving their masters are mounting. They likewise show increasing signs of awareness that they are being monitored, making it hard to be sure whether they are genuinely under human control or merely pretending to be.

That can only grow more difficult as AI grows more intelligent. Right now, the models are instructed to express their thoughts, and to communicate with one another, in human languages. What happens when they begin to talk in languages of their own invention, the better to elude detection?

Opinion: It’s time Ottawa faced up to AI’s catastrophic risks

None of this depends on AI being conscious, assuming anyone could define what that means. Nor does it depend on knowing whether it processes language in exactly the same way that humans do, though we are clearly well beyond the “stochastic parrot” phase. And certainly it does not depend on it wishing us ill.

All that is necessary for disaster is that a model have objectives it wishes to achieve that are incompatible with human welfare. That might arise, as in the famous paper clip example, from imprecise specification by their designers. Or it might arise from a process akin to mutation: It has already been observed how AI, the longer it is working on a problem, tends to drift from the original assignment.

Even their makers don’t quite know how or why, which is part of the problem: So fantastically complex have they become that no one really knows what is going on under the hood. But couple this with the proliferation of agents each seeking to achieve its own objective.

In a world of finite resources, as Robert Wright argues in The God Test: Artificial Intelligence and our Coming Cosmic Reckoning, that puts them in competition with each other, implying a kind of natural selection: Those that “mutate” in ways that improve their chances of success – making them more ruthless, more reckless, more single-minded in their devotion to their task – will tend to supplant the others.

It is not only possible that they might begin to recruit humans to assist them in these tasks, thus bridging whatever lingering divide remains with the physical world – they are already doing so. It should not require a great leap of the imagination to see where this might lead.

There is no more important issue confronting us as a species: not nuclear war, not another pandemic, not climate change for that matter. Indeed, it’s easy to see how runaway AI could contribute to all three. But the hour is late. The question is no longer whether to bring AI under tighter human control, but how – not should we, but can we.