Two years after the launch of ChatGPT, how has generative AI helped businesses?

ServiceNow Inc. is one of those gigantic companies whereupon scouring its website to learn all about its products and services, you might still wonder what, exactly, it does. This is possible, despite the fact the California-based company employs some 25,000 people and is worth around US$220-billion, unless you are familiar with the jargon-laden realm of information technology services, particularly concepts such as cloud enablement, workflow management and, of course, artificial intelligence for enterprise.

In November, ServiceNow held a large conference in downtown Toronto that was a paean to generative AI. Security was on hand to first check attendee credentials, lest anyone try to sneak into a session about configuration-management-database-as-a-service without properly registering. The morning kicked off with a video from ServiceNow chief executive officer Bill McDermott that played inside a darkened convention centre hall. “With AI, there’s a new mandate,” he shouted from beneath a tuft of grey hair. “It’s finally possible to do the things we’ve all been dreaming big about for years.”

Open this photo in gallery:

This video was followed by another one starring renowned actor and ServiceNow brand ambassador Idris Elba. “AI. It’s gonna change the game,” he said over a throbbing drum-and-bass soundtrack. “It’s time to lead. It’s time to put AI to work. It’s time to get to work.” He infused these lines with so much steely intensity that one almost felt compelled to leap from their seat and start driving customer value.

Inside of an expo hall, meanwhile, a variety of booths were set up to tell you how to do just that. Wandering around the room was like experiencing a LinkedIn feed turned corporeal. You could unlock the future of work with AI agents, enhance IT experiences with AI, and build no-code apps with AI. You could simplify procurement with AI or, at a separate booth, achieve next-level procurement with AI. It was the kind of place where a presenter could quip, “Does anybody here not have a workflow backlog?” and garner chuckles from the audience.

Now, here’s the thing: This is the unglamorous reality of generative AI today. Forget about superintelligence, killer robots and Nvidia Corp.’s soaring stock price. Generative AI is esoteric, a little dull, and some of the real-world applications can appear menial. But if you really want to understand the state of the technology and figure out whether it’s overhyped or transformative, whether a bubble is going to burst and lay waste to the entire industry or whether generative AI is going to be indispensable, you have to dive in and inhale the business process automation.

We are more than two years into a generative AI hype cycle that started with the release of OpenAI’s ChatGPT on Nov. 30, 2022. Proponents have promised many glorious benefits: efficiency gains, cost savings and supercharged productivity for individuals, companies and entire economies. “I firmly believe that AI is the holy grail of productivity,” federal Innovation Minister François-Philippe Champagne said at an event in November.

Open this photo in gallery:

There are plenty of naysayers who point out that generative AI is unreliable and prone to errors. Jim Covello, head of global equity research at Goldman Sachs, crystallized many of the doubts in a newsletter this past June. “Despite its expensive price tag, the technology is nowhere near where it needs to be in order to be useful for even such basic tasks,” he said in the report.

The reality on the ground is not so bleak. Companies are discovering there are many, many small use cases for generative AI: shaving off minutes and hours from a variety of tasks in areas such as coding, customer service, human resources, law and more.

There is a catch, though. These tasks are generally low-hanging fruit, the stuff that when automated or sped up does not always translate into huge cost savings or add to the bottom line, let alone filter up to the wider economy. A study released this fall by IT services company Kyndryl found that while nearly three-quarters of Canadian executives said they are investing in generative AI, only 41 per cent said they are seeing a positive return on those investments. “They’re struggling in Canada to find a net positive ROI,” said Stewart Hyman, Kyndryl Holdings Inc.’s chief technology officer in Canada, who chalked it up to the fact that some companies are just not ready from a technology and data perspective yet. But reliability issues could be a factor, too. “It limits the number of use cases where it’s easy to achieve a successful outcome,” he said.

Roger Premo, global head of strategy IBM, is seeing something similar with clients. Accuracy isn’t the only concern. “The second big obstacle that we’re hearing loud and clear is just the price performance of this,” he said. “The technology is very powerful, but for these use cases that are sometimes automating the kind of rote work within the business that’s also not the expensive labour.” In other words, the holy grail might be coming into view, but it’s still out of reach.

Open this photo in gallery:

Shannon Katschilo has a window into what companies are doing with generative AI in her role as the Canadian country manager at Snowflake Inc. The U.S. firm helps companies manage their data, including for AI. “A lot of organizations are taking a ‘crawl, walk, run’ approach,” she said.

Companies are starting with pilot projects for things such as chatbots to help workers retrieve information faster, or tools to summarize PDFs and other documents. Small potatoes, in a way. Indeed, a survey conducted by the MIT Technology Review and Snowflake found that only 30 per cent of executives view generative AI primarily as a revenue driver, and even fewer see it as a way to reduce costs. “There’s going to be a world in the future where the use cases are more revenue-generating,” Ms. Katschilo said. “But the large majority of AI is really working internally on the efficiency and productivity side.”

That said, small changes can still make a difference. Tim Brady, the chief executive of a Toronto-based e-mail and content management company called Colligo Networks, uses generative AI software for a range of HR functions. The software, made by another Canadian company called Borderless, has tools for onboarding, producing employment contracts, managing vacations and chatbots that can answer questions about, say, cross-border tax issues for international contract workers, the kind that Colligo frequently employs.

On that front, accuracy is paramount, but Mr. Brady said it hasn’t been a problem. “I’ve never thought about it, which is a good thing, because that means I haven’t had an issue,” he said. The biggest benefit of the software, he continued, is simply the time it saves. “I can just take all that complexity out of my life that I used to have, trying to pay contractors around the world,” he said.

The ability of AI tools to summarize large swaths of information is proving to be a benefit for Odaia Intelligence Inc. in Toronto. The company has been using AI for a while to help pharmaceutical companies identify patient populations for new drug treatments, and to find physicians who could prescribe those medications. More recently, the company has started using generative AI to summarize that information to help sales reps prepare for meetings with physicians.

Odaia has set up a voice interface, too, so that reps can get up to speed while literally driving to the next sales call. “That was completely wasted time before,” said Philip Poulidis, Odaia’s CEO. “Now you can have a conversation with the Odaia AI agent and ask it questions to help get ready for the next meeting.” That’s allowed some sales reps to squeeze in one or two more meetings each day.

At the Ottawa office of Vincent Baulne-Charland, the associate portfolio manager with BMO Nesbitt Burns has been using software from Toronto startup Boosted.ai for about a year. The company makes generative AI screening tools for finance professionals, pulling in recent news, trends, summaries of earnings reports and other data for stocks, industry sectors and countries. “The appeal is being able to analyze 100 or 1,000 times more data or news articles than we would be able to in a normal environment,” said Mr. Baulne-Charland, who works on a small team.

Putting hard numbers around benefits like this can be tricky. A lot of office jobs don’t have tangible, measurable outcomes. One exception is computer programming. Plenty of companies are using programs such as GitHub Copilot, which offers coding suggestions as developers write script. In a study published in September, researchers from the Massachusetts Institute of Technology, Princeton and other institutions examined use of the tool among more than 4,800 software developers. Copilot increased the number of tasks programmers completed by 26 per cent, and the least experienced coders benefited the most.

Open this photo in gallery:

The virtues of generative AI can depend on how far companies are willing to go. Neo Financial Technologies Inc., a fintech company based in Calgary, has deployed a generative AI customer service chatbot using technology from Toronto-based Ada Support Inc. The chatbot has been able to deal with between 50 per cent and 60 per cent of customer problems, without escalating to a human agent. “It’s actually learning very, very fast,” said Neo co-founder and CEO Andrew Chau.

Neo employs around 150 customer support staff that can handle the problems that AI cannot. In some cases, that’s not because the technology is too unreliable, Mr. Chau said. Take chargebacks, for example, in which a credit card holder disputes a transaction. The AI bot can handle part of the process. But third parties such as credit card companies are involved, and they have their own policies and procedures that are not yet compatible with Neo’s AI approach. “But you can start to chip away at what’s needed,” Mr. Chau said.

Larger companies are typically more cautious about generative AI. Toronto-Dominion Bank is gradually deploying a chatbot to its 1,200 call centre workers by helping them retrieve answers faster. The early results are promising, with a 15-per-cent reduction in hold times, and another 15-per-cent drop in the number of times a customer service rep has to speak to their managers to get an answer. To guard against the chatbot making things up and providing incorrect information, the assistant provides a link to source material, which call centre reps are supposed to verify. “We’re trying to keep the human in the loop as our checking mechanism, so that the customer gets a really good quality answer,” said Luke Gee, TD’s head of AI and analytics.

Putting that bot directly in front of customers, as Neo Financial does, could theoretically provide more benefits. But Mr. Gee isn’t quite sure about that. “That will be interesting to work out, whether we can get to a point where it is accurate enough,” he said. “I would still be more comfortable with the human-in-the-loop at this point.”

Open this photo in gallery:

Some 700 companies, meanwhile, are using technology from Quebec-based Coveo Solutions Inc. on their websites and other digital platforms. Founded in 2012, Coveo uses generative AI to power the help sections of the websites for Dell Technologies Inc., United Airlines and others, and recently announced a deal with Shopify Inc. to bring its search and question-answering tech to merchants.

Travel and expense company SAP Concur has been able to reduce customer service case loads by 31 per cent thanks to Coveo, while accounting software company Xero Ltd. saw a 20-per-cent boost in customers resolving issues for themselves within the first six weeks. “Customers are now recognizing what companies like us in particular – and there are not that many – can do in terms of having a real ability to deliver generative AI at scale,” said CEO Louis Tetu.

I threw a few queries at the Coveo-powered United Airlines help page, and found it was a microcosm for the state of generative AI. It aptly fielded questions about gluten-free meals, seat belt extenders and travel with pets, while deflecting nonsense queries about flying with my emotional support horse.

Then I hit turbulence. I was told I could travel with swords, knives and other sharp objects, provided they were securely packed. I asked again, but I was told, no, I could not bring a sword. When I asked how many swords I could bring, I was told I could bring a single bamboo sword. Other times, depending on how I phrased the question, the site didn’t answer at all.

Coveo said that it could not replicate my exact experience. Laurent Simoneau, the company’s co-founder and chief technology officer, said in an e-mail that similar questions could yield slightly different answers, depending on the material the AI application retrieves, and the relevance of any particular topic. But as the technology improves, he added, the number of times an LLM provides no answer at all should fall, “reflecting a growing ability to answer a broader range of queries.”

Open this photo in gallery:

The big question is when all of these tentative implementations are going to scale up and amount to something meaningful. Estimates about the economic impact of generative AI are all over the map. Microsoft Corp. has said generative AI could add $187-billion annually to the Canadian economy by 2030, pushing productivity growth from 0.6 per cent to an “astounding” 8 per cent, if we act quickly. Google, meanwhile, has said it can add $230-billion to the economy and save the average Canadian worker over 175 hours a year.

Daron Acemoglu, an economist at MIT, is more conservative. He’s estimated the technology will increase productivity in the United States by only 0.5 per cent cumulatively in the next decade, mostly because it can’t handle complex problems and is limited to rote, menial labour.

“I’m skeptical of anybody who says they can provide a number,” said Avi Goldfarb, a University of Toronto professor and Rotman Chair in Artificial Intelligence and Healthcare. His own thinking on the question is much broader, and key to answering it is whether AI proves to be a general-purpose technology. Think electricity, the steam engine or the internet – developments that had a huge impact on how we work and live by spawning new products, services, companies and business models over time. “The trouble is you don’t really know if they’re general-purpose technologies until it’s all played out,” he said.

There are signs that AI – or, as Prof. Goldfarb thinks of it, technologies that use data to support decision-making – fits the bill, in part because it is being adopted across industries and carries the potential to spur further innovation. “Whether it’s going to be big enough to have this outsized impact on productivity remains an open question,” he said. “I do think of the technologies we have out there, it’s by far the most likely.”

Some evidence shows that recent adoption of AI by Canadian companies has been a bit of a wash. The Dais think tank at Toronto Metropolitan University recently looked at companies that implemented AI between early 2020 and late 2021 and found no strong effect on short-term productivity – positive or negative. The companies that adopted AI were already more productive than their peers, according to a December report. There are limitations – the time period does not capture the recent advances in generative AI – but the results are noteworthy. “The findings do call for caution in presuming that business adoption of AI can be a silver bullet in addressing Canada’s productivity growth challenge in the near term,” the authors wrote.

Open this photo in gallery:

In the meantime, the costs of generative AI are rising, which could further blunt its impact. It is already tremendously expensive to train an advanced large language model (LLM) – the type of model underlying generative AI applications – owing to the sheer number of graphics processing units needed, the time involved, and the post-training tinkering that is required. The price tag has risen by about 2.5 times every year since 2016. By 2027, the largest models will cost more than US$1-billion to train, according to research organization Epoch AI.

Longer-term, it could be more expensive for customers to use LLMs, too, a process that is called inference. Typically, companies access LLMs by paying for each word, or part of one, that is produced by the model. That costs just fractions of a penny today, and while the price has been falling, the decline could be temporary.

The economics of inference didn’t matter all that much until a couple of years ago, because hardly anyone was using generative AI. The more recent surge of interest compelled companies to figure out how to improve the efficiency of inference and lower costs. “That will still continue for a little bit, but then at some point you’re going to hit this floor, and cost is going to become more dictated by model size,” said Jacob Steinhardt, the cofounder of AI research lab Transluce.

Companies are still trying to build bigger, more powerful AI models in hopes of unlocking new capabilities. Those models inevitably come with higher costs that could be passed on to customers. Mr. Steinhardt estimates that inference costs will rise from about 20 US cents an hour today to US$2.50 by 2030.

The staggering development costs naturally raise concern about how LLM developers such as OpenAI, Anthropic and others are going to turn a profit. “It’s definitely a resource intensive industry,” said Nick Frosst, a co-founder of Cohere Inc. in Toronto, which builds LLMs for use by businesses. “There’s a lot more value we can get out of LLMs without them changing at all.”

Open this photo in gallery:

With the right tweaks and customizations, and pointed at the right problems, generative AI will become more and more useful to businesses, he continued. “We can make tons of value, tons of useful things by continuing to iterate on the tech and making breakthroughs that have nothing to do with just throwing money at the problem,” Mr. Frosst said. “We’ve been a lot more capital efficient than some of the other players in the space.”

Huge models aren’t everything, either. LLM developers are also rolling out smaller models that can be just as good as their larger brethren for some specialized tasks. Plus, they are cheaper to build and use. “It’s pretty profound. It’s like the performance of the very large proprietary models at 95 per cent lower cost per inference,” said Roger Premo at IBM. “The pairing of smaller models with more advanced tuning techniques will start breaking this price performance compromise.”

ServiceNow, for one, is putting that premise to work. The company, which bought Montreal-based Element AI in 2020 for US$230-million, uses its own proprietary models and open-source models (read: free) that it tailors for specific applications. “We are running the majority of the use cases on our own models and our own data centres. We can have a very, very tight grip on the cost structure,” said Nicolas Chapados, the company’s vice-president of research, who co-founded Element AI.

Its big AI play is a platform called Now Assist that allows businesses to access tools for coding, summarizing, producing content and to aid IT, HR and customer service reps resolve issues faster. Internally at ServiceNow, the platform has resulted in US$10-million in annualized financial benefits, a combination of cost savings and productivity gains, across some 50 use cases. (For context, ServiceNow earned more than US$8-billion in revenue in the first nine months of 2024, and had close to US$5.4-billion in operating expenses.)

Chris Ellison, the company’s general manager for Canada, could not comment on how much ServiceNow had invested into its AI platform and whether it was profitable, but said it has been the fastest-growing product in the company’s history. “Customers are voting with their wallets. They’re voting with confidence,” he said. The company has 44 customers spending more than US$1-million on Now Assist.

We were having this conversation in a meeting room at the ServiceNow conference, where down the hall, sessions were getting under way about AI-generated customer service case summaries and AI-powered workflow solutions and talent strategies. It served as a reminder that should that glorious day of unbridled productivity arrive, it will be built on some of the least exciting things imaginable.