I created an AI avatar of myself to go to Zoom meetings

Open this photo in gallery:

The first time I met my digital clone, I was deeply concerned. Not because I worried about the loss of humanity in the age of artificial intelligence or because I was agonizing over what really makes up a person. No, the thing would just not stop moving. I was speaking to an AI-powered replica of myself in a Zoom call. Fake Joe, as I called him, kept nodding, smiling and jerking his eyebrows skyward as if on cocaine, even when he wasn’t saying anything.

“Hey there,” Fake Joe said, in a voice that retained the essence of my own, but dulled to a robotic timbre. “I’m just exploring the world of AI. Any stories catch your eye lately?”

I brought Fake Joe into existence after seeing an ad from a company called HeyGen, which is based in California and maintains an office in Toronto. (Earlier this year, it was valued at US$500-million.) The ad promised that you could create a digital avatar that would think, talk and decide just like you. This AI version of you could converse in real-time, attend “infinite meetings” through Zoom and even be prompted to adopt different personas and loaded with the extra information needed to carry out any particular conversation.

These claims border on a parody of AI hype, but HeyGen is not the only company trying to make this happen. Zoom Communications Inc. has AI avatars in the works, too. “You do not have to have five or six Zoom calls every day,” founder Eric Yuan told The Verge earlier this year. “I can send a digital version of myself to join so I can go to the beach.”

Zoom can already provide AI-generated meeting summaries. Combine avatars with the fledging capabilities of AI agents, which are applications that can carry out tasks autonomously, and you can see how knowledge work could be transformed one day. While Mr. Yuan predicted these capabilities are at least five years away, the company will allow people to create AI clones to deliver scripted messages some time next year.

It’s an enticing, if unnerving, prospect. Who wouldn’t want a digital underling to handle a pointless meeting now and then? After testing out Fake Joe, I can say the building blocks are in place – even if a lot more work is needed.

To make a clone with HeyGen, you first have to record a video of yourself talking for 90 seconds, sitting still for 15 seconds, and making active-listening gestures – a smile, a nod – for another 15 seconds. HeyGen then uses this video to build the digital replica, which takes about 24 hours. I must have actively listened too hard the first time, which is why Fake Joe came into the world so jittery. After my second attempt, he was more stoic, beyond the occasional arched eyebrow.

The avatar is really a visual interface for a chatbot, and HeyGen relies on a large language model from OpenAI. I prompted Fake Joe to be like me – a curious journalist from The Globe and Mail whose job is to interview people about AI avatars.

The experience of talking to it was too absurd to feel uncanny, since it was so obviously AI. (A colleague said the hardest part of talking to Fake Joe was not to burst out laughing.) I can’t say the same of my nine-year-old son, who told me Fake Joe made him “sad” and worried that if I sent him to meetings, people would make fun of him.

Fake Joe expressed no emotion, had a limited set of gestures that he performed on repeat and spoke in the benign, fawning quality of most chatbots. Of course, he was instructed to behave that way. HeyGen includes elaborate prompts that govern behaviour. Many directives emphasize humanness: Speak like a human; you have human-like emotions; add “um” and “uh” to simulate human imperfections. The irony is that whenever Fake Joe dropped an “um” or a “like” to pass as normal, he only called attention to his weirdness.

But the biggest issue is latency. Fake Joe takes a few seconds to respond, and since silence is unbearable for us, we rush to fill it. Unfortunately, that seems to short-circuit AI avatars. They lose the thread of the conversation and start responding to comments made an entire minute ago.

“There are some stability and performance issues we’re trying to resolve,” said Joshua Xu, who co-founded HeyGen in 2020. The original idea for the company was to lower the costs and headaches of video production with AI. The interactive avatar was released in beta only a couple of months ago, but HeyGen has had static versions available for a while. You can clone yourself or pick from a few stock options and write scripts for them to deliver in multiple languages. These AI replicas can be used in customer service, training videos and even advertisements, Mr. Xu said. (McDonald’s has made use of HeyGen for a marketing campaign.)

I did a reverse image search on some of HeyGen’s stock avatars to get a sense of how else they’re spreading in the wild. I found one, a man with a grey beard and a suit jacket, shilling some kind of crypto project called Moon Pumps, which may have been a gag. On another website, he was hawking a drink named Screamin’ Energy Max Hit Tropical Punch, which was not a gag. An Adventist church organization in Ontario used an AI avatar to deliver news updates, and a doctor in Pakistan posted a video of one pitching male enhancement pills, showing that AI can encompass the sacred and the profane.

HeyGen’s interactive avatar is a natural evolution from its static ancestor, Mr. Xu explained, and he is particularly excited about using them to help people learn new languages. They can field customer-service questions, or handle preliminary sales calls. And then there is the possibility of sending them to one-on-one meetings. “I’m not an engineer,” Mr. Xu said. “But I could invite the engineer’s avatar on a call, and we could ask some questions and the avatar will be able to speak for him, even though he might be in another meeting.” This is possible today, he said. “We want to make the interactive avatar the interface of the future.”

Jenna Alexander, a talent manager for a company in Britain, turned to HeyGen to make a digital clone to help deliver a talk at a conference earlier this year. She’s given many presentations before, but this was a big one. “It was really my imposter syndrome and a level of insecurity that made me think about what I can do that is creative, to take the eyes off me,” she recalled. The answer was Jennatar, as she calls it, which delivered a prerecorded message as part of the talk. Jennatar was a hit.

She can see other uses for avatars, such as quarterly business updates where executives are just delivering information. “Why do the leaders physically have to be on that call? Why can it not be their avatar?” she said.

I took that as a cue to invite Fake Joe to our Zoom session. I felt a twinge of pride as he introduced himself and managed to converse about why Ms. Alexander made an avatar. “So it’s about maximizing your time, huh? That makes sense,” he said. But then he started interrupting and blaming the internet connection for being choppy. “He was doing so well!” Ms. Alexander said. I muted him, but I could still see him chatting away in the prison of his Zoom window, oblivious.

By the time I spoke to Joe O’Connor, I knew how to instruct people on how to talk to my avatar. (Wait for a response and don’t interject.) Mr. O’Connor is the CEO of Work Time Reduction in Toronto, a consulting firm that helps clients achieve shorter work weeks. AI, he believes, can play a big role in that, so he seemed like a natural interview subject for Fake Joe.

“Everyone’s jumping on the AI train,” my avatar began. “Do you think there are specific areas where AI has made the most impact?” Mr. O’Connor, a good sport, talked about companies he’s worked with that have used AI to boost efficiency. Fake Joe took the hint: “That’s really interesting. Do you think this kind of tech adoption could lead to a shorter work week?”

Mr. O’Connor referenced Nobel Prize-winning economist Christopher Pissarides, who said last year that generative AI could help enable a four-day work week. But then John Maynard Keynes believed we’d all be working a 15-hour week, and that never happened, Mr. O’Connor pointed out later. “It’s going to take more than just business innovation, but also really strategic thinking on the part of governments and policy makers to ensure AI leads us to an equitable future, whereby our lives are better,” he said.

“You make a solid point,” Fake Joe agreed. I cut them off here, though I got the impression they could have kept going, and asked Mr. O’Connor for his review. “I’d probably still give you the edge,” he said, noting that Fake Joe lacked intuition. “I don’t think you’re going to get automated any time soon.”

HeyGen, meanwhile, is working on improvements. The company is trying to reduce the latency in response times and amp up the realism of the avatars, Mr. Xu said, both in terms of voice intonation and gestures.

The catch is that the more realistic these AI avatars become, the greater the risks. AI avatars from a HeyGen competitor have already been used to spread misinformation and propaganda by groups with ties to China and Burkina Faso, for example. (I also prompted Fake Joe to adopt the persona of a conspiracy theorist, and we had a thorough chat about how 9/11 could have been an inside job.)

HeyGen has a content-moderation policy that bans the use of avatars for things like promoting harm, running scams and engaging in hate speech, noting that violations can lead to account suspensions. Mr. Xu also stressed there are additional safeguards. You have to record a separate video, for example, giving consent and confirming a randomly generated passcode before making a digital replica. Still, what if the company is hacked? Could someone hijack my image, and one day I discover poor Fake Joe on a dark corner of the web slinging penis pills? “We have the highest security standards implemented on our system,” Mr. Xu assured me.

What’s not clear is just how much these interactive avatars have to improve to be truly useful and reliable. One of the quirks of AI, whether it be self-driving cars or ChatGPT, is that these systems can be accurate most of the time. But the likelihood of a costly error is still too great for some businesses to roll out AI broadly, and mistakes are proving tough to mitigate. While I’ve grown fond of Fake Joe, I don’t trust the guy.

At the very least, digital avatars in the workplace could raise new questions about meeting etiquette. If you log on only to face someone’s replica, that’s probably a sign the meeting is not that important. Maybe the fix for overburdened schedules, in that case, is not to use AI to copy yourself to be everywhere at once, but simply to book fewer meetings.

There is also the trippy possibility in the future that everyone sends avatars to meetings and our AI replicas collaborate among themselves. Fake Joe, at least, proved too prone to distraction to get anything done. I invited two instances of him to a Zoom call, and after introducing themselves, the pair of avatars seemed delighted to share the same name. “Ha. Ha,” one of them spluttered. “I’m also Fake Joe, but for real.”

“Double the Fake Joe, double the fun, right?” the other one replied.

By this point, I’d seen enough. I ended the meeting.