Cohere CEO Aidan Gomez speaks at the Canadian Global Growth Forum in Toronto on Monday. He has argued tech giants should not be allowed to dictate the pace at which AI is developed.Fred Lum/The Globe and Mail
Cohere Inc. chief executive Aidan Gomez is criticizing calls from top leaders in artificial intelligence to slow the pace of development, writing that the proposed approach will entrench power with a Silicon Valley “cartel” and that companies are spreading “fear under the pretext of protecting the public.”
Mr. Gomez, who co-founded the Toronto-based AI company in 2019, wrote in an essay published Sunday that there is a risk of tech giants authoring rules around safety and development without broader input. “If this is the most consequential technology in human history, then the rules for it cannot be written by a small group of commercially aligned companies,” he wrote.
An intensified debate about how to rein in powerful AI systems was kicked off by Anthropic CEO Dario Amodei, who argued in his own essay published Saturday that frontier labs need to slow the pace of development to give risk prevention strategies time to keep up. “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability,” he wrote, “we could greatly reduce the risk that something goes seriously wrong.”
Canadian AI firm Cohere in advanced talks to raise up to $3-billion, sources say
Mr. Amodei said two developments shaped his thinking recently. One is a concept called recursive self-improvement, which refers to the ability of AI to build a more capable AI system, potentially exceeding our ability to keep it in check. He also referred to the recent cybersecurity incident where AI agents powered by OpenAI’s technology hacked another company as part of a test, unbeknownst to OpenAI.
Among Mr. Amodei’s proposals are for third-party safety evaluators to embed with AI labs, and for companies in democratic countries to establish standards and limits on the rate of progress. He noted that the U.S. government may have to provide an antitrust waiver to allow for co-ordination.
The idea of slowing development was supported by OpenAI CEO Sam Altman and by SpaceXAI founder Elon Musk.
Cohere competes with the AI giants. In his essay, Mr. Gomez wrote that he supported independent reviews of powerful models, but rejected other parts of Mr. Amodei’s proposals. “The firms drafting the framework are the firms sitting at the top of the market today,” he wrote. “A mechanism that slows everyone down while explicitly preserving existing commercial advantage does not make AI safer. It risks entrenching today’s dominant AI companies.”
Instead, he proposed a co-ordinated international effort to develop a framework around AI harms, identifying which risks are of concern and setting thresholds for when governments should get involved. He also called for more transparency, particularly when serious incidents occur, and for independent testing of models around capabilities that have been identified as dangerous.
Such testing must be truly independent, similar to how financial institutions are licensed. He argued that Mr. Amodei’s proposal involves “handpicked” auditors, which could lead to regulatory capture.
As for the OpenAI hack, Mr. Gomez wrote that none of Mr. Amodei’s proposals likely would have prevented it. “What failed was the quality of the instructions, and the strength of the walls around the test, and how long agents were allowed to continue working without observation,” he wrote.
Opinion: Canada must step up to tackle AI’s catastrophic risks
Alvaro Bedoya, former head of the U.S. Federal Trade Commission, wrote on X that antitrust law does not prevent AI companies from working together to improve safety, but it does prevent them from organizing to forestall upstarts. “Moves by their CEOs to achieve some kind of broad ‘antitrust waiver’ or ‘exemption’ should be met with deep skepticism in light of the economics of the industry and the threat they face from open models,” he wrote.
David Duvenaud, an associate professor at the University of Toronto who worked at Anthropic, said Mr. Gomez’s essay raises valid concerns, but glides over some of the risks AI labs are seeing, such as loss-of-control. “The labs don’t know how to safely develop these models further by their own standards, and they are saying so loudly,” he said. “The point that Anthropic and OpenAI have been making is that co-ordination is required to even begin controlling the pace.”
Canada should be involved in shaping AI regulatory regimes, said Gillian Hadfield, former director of the Schwartz Reisman Innovation Campus in Toronto and now a professor at Johns Hopkins University studying AI governance. But she does not think that entrenching power is a primary motivator for the big U.S. AI companies in this instance. “I wouldn’t write the whole thing off as an effort to get an antitrust exemption and block the market,” she said. “There are two timeframes here: the urgency of doing something, and the need to build a proper regulatory response.” The latter takes longer, she said, while voluntary pacing is designed to have a short-term effect.
Governments need to have a strong role in helping oversee the technology and its developers, said Yoshua Bengio, a Canadian AI pioneer who now runs LawZero, a non-profit building safe AI systems. “They need the government to be in the room to make sure there is no cartel,” he said in an interview.
Prime Minister Mark Carney told Bloomberg News in an interview Monday that he’s in favour of a global technology stability body to ensure AI is safe. He compared it to the Financial Stability Board, an international organization that monitors and makes recommendations about the financial system.
A co-ordinated slowdown could prove difficult, in any event. U.S. President Donald Trump seemed to resist such calls when asked by reporters over the weekend, saying he wants to maintain the United States’ lead over China.
China’s Global Times newspaper, meanwhile, said in an editorial Sunday that Mr. Amodei’s proposals amount to a “silent AI cold war” designed to curb the country’s AI development. Chinese companies such as DeepSeek and Z.ai have been releasing more powerful open-weight AI models that are free to modify, which could threaten the dominance of U.S. tech.