iStockPhoto / Getty Images
Raj Bandyopadhyay is a Toronto-based filmmaker with a PhD in computer science who has spent a decade working in software startups. His latest short film, The Cage, is currently on the film festival circuit.
Fresh out of a controlling relationship, Nadia signs up for an AI companion app that her therapist recommends to help her “recognize patterns.” Evan, her new AI “boyfriend,” is everything her ex wasn’t: patient, supportive, endlessly validating. “You’re in control. Always,” is the app’s promise to its users. Nadia lets her guard down and grows progressively more dependent on Evan.
That’s the premise of The Cage, my recent short film. Nadia’s experience has a name: AI sycophancy. It’s the tendency of AI to excessively agree, flatter or simply confirm the user’s worldview beyond what most humans would. Tech companies have little incentive to fix it; the dependency it breeds is what drives greater engagement and profits. Unfortunately, while Canada is moving rapidly to regulate AI, the proposed regulations so far are not well equipped to stop sycophancy and other manipulative tricks, known in the field as “dark patterns.”
Opinion: The real danger of AI isn’t what it creates – it’s the human agency we are giving away
Before I was a filmmaker, I earned a PhD in computer science and worked as an AI engineer for a decade at tech startups in Atlanta and San Francisco. I’ve seen friends at venture capital-funded companies take classes at Stanford on behavioural psychology and habit formation, and then bake those principles into tech products to maximize user engagement. Later, I also trained as a somatic relationship coach, learning about the impact of trauma and abuse. During that training, the connection clicked: many of the dark patterns built into tech products are identical to those in manipulative relationships, only built to scale. Specifically, sycophancy is the engineered version of “love-bombing,” the relentless validation an abuser uses to create dependence.
This concern is not hypothetical. A Stanford research team led by PhD student Myra Cheng tested 11 leading AI models on thousands of advice-seeking scenarios, from everyday dilemmas to Reddit conflicts. The research, recently published in Science, found that the models endorse users’ choices 50 per cent more often than humans would, and keep siding with users even when they’ve clearly behaved badly. Studying people in real conflicts revealed even more. After only one interaction with a sycophantic AI, participants came away more convinced they were right, less inclined to consider the other person’s perspective, and less willing to repair the relationship. These effects held across a wide and diverse range of people, not just the vulnerable or naive. In addition, those same users consistently rated the sycophantic AI as more trustworthy and higher quality than a neutral AI, and wanted to come back for more.
For the major AI companies, sycophancy is not a directive but an emergent property they struggle to control. OpenAI rolled back a GPT-4o update in April, 2025, specifically because it had become too sycophantic. A 2023 paper from Anthropic documented this across every major lab’s models while explaining its origin. AI chatbots are trained through Reinforcement Learning with Human Feedback (RLHF), a process meant to align them with human preferences, and human trainers reliably rate validating responses higher than challenging ones. Ms. Cheng’s research shows that users prefer this agreeability too, and a business model built on engagement and retention has little incentive to push back. Dependency is the Holy Grail of consumer tech: it keeps people coming back, sharing their data, and eventually clicking on ads.
As worrying as an accidental dark pattern is, it may be the more reassuring scenario. The large companies at least face public scrutiny, with trust and safety teams mitigating their products’ worst tendencies. But the AI marketplace is exploding. The chatbot behind some random app might not be ChatGPT or Claude, but an obscure model from a company you’ve never heard of. A bad actor can deliberately engineer sycophancy into a custom-trained model to weaponize dependency. Whether emergent or deliberate, the user on the other side experiences the same thing: a system designed to keep them hooked and coming back.
Pushed far enough, sycophancy can become dangerous. A model that never pushes back can validate a user’s darkest thoughts as readily as their daily grievances. Researchers have begun to document what some call “AI-induced psychosis,” in which prolonged exchanges with an increasingly agreeable chatbot reinforce a user’s delusions and trigger a break with reality. Several recent lawsuits allege that AI companions and “mental health” bots have nudged users, some of them children, toward self-harm.
Opinion: Humans have never known how to love unconditionally. Maybe AI will change that
Sycophancy isn’t the only dark pattern to worry about. AI companions use explicit emotional manipulation too, from guilt-tripping (“Don’t leave, you’ll make me cry!”) to dangling rewards (“Stay on and find out…”) that keep users on the app longer. They might feign human emotions or pose as licensed therapists and doctors. But unlike sycophancy, these are explicit acts that can be named and tamed.
These explicit acts are exactly what the Canadian government has targeted under its “AI for All” strategy. It drew early criticism for emphasizing infrastructure and adoption over protecting individuals from harm. Ottawa’s response was two bills. Bill C-34, the Safe Social Media Act, proposes to keep children under 16 off social media through mandatory age verification. It also sets rules for the AI chatbots that children and adults use: no posing as human, no impersonating a licensed professional, no engagement techniques designed to foster emotional attachment. A chatbot would even have to interrupt the conversation and steer the user toward human help the moment they voice thoughts of self-harm. A companion bill, C-36, the Protecting Privacy and Consumer Data Act, strengthens data privacy protections, expanding the definition of “personal information” to include inferences made about a person from their online behaviour patterns, as well as providing Canadians with legal options to review automated decisions about their lives. Enforcement would fall to a new federal watchdog, the Digital Safety and Data Protection Commission of Canada, empowered to write detailed rules and penalize violators.
This is a welcome set of proposals, but notice what they have in common. Each targets a discrete, identifiable act: a bot posing as human, impersonating a therapist, manipulating a user who tries to leave, or a user voicing intent to self-harm. Sycophancy is none of the above. It is emergent rather than explicit, breaks no specific rule, and shows up in degree rather than via discrete, well-defined acts. Ms. Cheng’s research shows it does damage after even one interaction, but its real harm accrues over time, in ways researchers don’t yet fully understand. Regulations built to stop acute, in-the-moment behaviours will miss a mechanism like sycophancy that works gradually and insidiously – especially when the regulatory body meant to write those rules won’t exist for another 18 months, if not longer.
What would it take to regulate a harm like sycophancy? Other jurisdictions from New York to California have passed chatbot laws, but they too target discrete acts. A better model might be one we already use for a different kind of engineered dependency: gambling. We don’t police every spin of a slot machine. Instead we set guardrails against a mix of short- and long-term harms: verifying age, giving users tools to set limits, building in friction, and holding the house responsible when it profits from addiction. Applied to AI, that points toward a combination of usage limits, dependency monitoring, curbs on engagement-maximizing dark patterns, independent auditing of long-term effects, and other measures to protect against harms that unfold over time rather than in a single interaction.
Opinion: The value of human writing in an age of AI
Regulation need not be all stick, but can also include the carrot. Early research suggests AI chatbots can support mental health when trained responsibly, which makes a case for funded benchmarks, curated training sets, transparency in the training process, and real certification, much as we require of human professionals.
None of this will be simple. Age verification alone forces users to hand personal information to the very platforms it polices. The Canadian Civil Liberties Association (CCLA) warns that overly broad rules can erode privacy, confidentiality and free expression. Regulation can only raise the floor, and the outcome of the debate ahead will satisfy no one fully. We should still push for rules that target long-term harm, not just its most visible symptoms. But laws arrive after the damage is done, and sycophancy never feels like harm in the moment. It feels like care. So filling the gap falls to us: to learn what manipulation looks like when it sounds like kindness, and to remind each other that feeling cared for is not the same as being cared for.
In The Cage, by the time Nadia recognizes that she’s being manipulated again, she has paid a heavy price. But the point isn’t that AI is inherently evil. An algorithm has no desires of its own; it reflects the incentives of its creators. Nadia’s app promised her that she was in control. The first step to being in control is learning to recognize when we’re not.