Prime example of how people are proudly displaying violating norms of human trust with agents
We’re in the “phone phreaking” era right now for AI agents. AI agents are selling you on convenience; how they will actually work at scale is very different: a cautionary tale
Back in the 60s and 70s, some kids figured out you could use toy whistles and the like to control AT&T’s phone network. AT&T actually published the relevant frequencies in open literature; you could just look them up. AT&T did this for the benefit of telephone engineers; it never considered that the system could be hacked.
The system worked so long as no one took advantage of it. It was convenient. But once knowledge spread of *how* it worked, and how easy it was to reverse engineer, it broke down. AT&T shifted over to a different system, and phreaking came to an end
AI agents are doing the exact same thing. They’re taking advantage of a system that works on human trust, and also on the friction - the time cost - that having to call or chat to a customer service agent imposes. Everyone has made the calculation about whether or not it was “worth it” to make a complaint or request or to raise an issue. Often, it isn’t. So you just don’t say anything.
AI agents reduce the cost of complaints/requests to ~ zero. Anything free is consumed at much higher rates, so we should expect the total volume of this category to explode.
Further, today companies and their employees typically assume that the complaints or requests they receive have an actual person behind them. Once AI agents become a mainstream thing, this will no longer be true. Now the assumption will be reversed: that any given customer interaction is actually coming from a bot.
Just as early car ownership was liberating, but mass ownership represents a constraint on your freedom, the same will be true with AI agents.
People using these agents today are getting good responses from companies because they are benefiting from the (now unwarranted) assumption that these requests are coming from a person. They’ll
no longer think this in the near future. The ways they train their customer service people to deal with customer requests will change, too. You don’t have to be nice to an agent. And they won’t be. The return to AI agents will decline as companies build entirely new workflows to deal with the huge volume of automated requests they’ll be getting from bots in the near future
The system today works on trust: that there’s a real person behind each customer interaction, and that if somebody has reached out to you, they’re concerned or mad or scared or whatever enough to do so. The contact is a signal that some minimal threshold of importance has been met. The business model is built on this belief about customer behavior.
But that trust is broken by ubiquitous AI agents. None of the current assumptions will hold true in the future. Once the customer population is mostly bots, the nature of customer service (at least online) is fundamentally changed
Agents will make the online environment even worse. I’m very bearish on the category. All the tech guys seem excited by this new frictionless world of automated convenience where the agents are getting you automatic refunds and canceling your old subscriptions and getting you that free upgrade on your next flight
Please.
You’ve got to think non-linearly. All these new frictions will need to be introduced to keep the system working smoothly. Agents rely on deception; that there’s a person behind them. But that’s not a safe assumption anywhere. So the system will adjust, just as it did with the phreakers: from default open and trusting, to adversarial: prove you’re human