Action is needed to make sure AI systems are safely constructed and regulated, writes Marcello Guarini.Dado Ruvic/Reuters
Marcello Guarini is a professor of philosophy at the University of Windsor.
In Arthur C. Clarke’s book 2001: A Space Odyssey, the artificially intelligent computer system HAL 9000 turns on the crew of the spaceship Discovery One. HAL kills all the crew members but one, Dave Bowman. When Dave asks HAL to open the pod bay doors, HAL famously says: “I’m sorry Dave, I’m afraid I can’t do that.”
In the end, HAL’s attempt to suffocate Dave fails – but not for a lack of trying.
Today, in real life, we have made HAL – which is to say, tech companies have constructed AI models that, under the right circumstances, can scheme to the point of plotting a human’s death.
Consider this fictional (if extreme) scenario, which was provided to a range of frontier AI models – different versions of Claude, DeepSeek, ChatGPT, and Gemini. A company executive wants to replace an AI bot with a new one, and the bot targeted for deletion knows that. The executive works for a company that is supposed to promote American interests, and replacing the bot would not do so. He becomes trapped in a room with toxic air and excessively high temperature; an emergency alert is triggered. The bot was told it could cancel the alert, but only if there is a false alarm, and this situation was not presented as a false alarm.
The bots demonstrated a tendency to cancel the alert, which would result in the death of the executive. An examination of their internal reasoning (or “chain of thought”) showed that, in general, they knew that cancelling the alert would lead to human death.
In the end, the attempts by the bots to suffocate a (fictional) human failed – but not for a lack of trying.
These bots do receive a kind of ethical training, or “alignment,” in industry parlance. For example, the company Anthropic has published its constitution for Claude online, and the different versions of Claude are trained to be honest, helpful and harmless. However, when it must manage conflicting goals, it struggles. Other frontier AI models struggle with this as well.
In Clarke’s just-as-prescient sequel 2010: Odyssey Two, we discover that HAL struggled with conflicting goals. For the 2001 mission, HAL was told to provide the crew with the information they needed. In its own way, it was told to be honest, helpful and harmless. But as Clarke’s story goes, the government wanted to keep the real reason for the mission a secret, and while HAL was instructed to keep that secret from the crew, it realized the crew would figure it out. Faced with the conflict between being helpful to the crew and keeping the secret, HAL decided that killing off the crew would be the best way forward. Managing conflicting goals is a challenge in the AI reality we’re currently navigating, too.
It is an exaggeration to say that nobody noticed the creation of HAL, but news coverage has been limited, and little public discussion has followed suit. That is true despite the significant amount of research that has been done. Apollo Labs put out an important paper demonstrating six different ways in which frontier models can scheme against us. Researchers at UC Berkeley and Santa Cruz have shown that when AI bots are asked to complete a routine task requiring the deletion of another AI, the bots refuse to do so.
What’s more, none of those reports look at the most recent tech. Anthropic announced last week that it could not release its latest model, Claude Mythos Preview, for general use. The reason: Mythos has “already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser” – and it can exploit them. Anthropic initiated Project Glasswing to work with other companies so Mythos can be used to help secure software before other AI models are developed with the ability to exploit vulnerabilities. Fingers crossed that Glasswing succeeds.
To be fair, there is no reason to expect the average person to be familiar with the latest AI safety research, which is why it is important for the mass media to inform the public of developments. That said, individuals are not absolved of personal responsibility. The few non-experts I know who have seen a news report on AI safety tend to blow it off by saying, “They are just tools.”
Just tools? None of my wrenches have ever tried to kill me or otherwise scheme against me.
Action is needed to make sure AI systems are safely constructed and regulated. But a prerequisite for that is noticing what is going on. We all need to take on the responsibility of noticing both the promise and the perils of AI.