I’ve just gone through Anthropic’s September threat intelligence report and there’s something in there I think people are underestimating.
We already know hackers are using AI. That’s not really news anymore.
What caught my attention is how little the human sometimes needs to do now.
Anthropic documented operations where AI was being used across reconnaissance, exploitation, credential harvesting, processing stolen data and even modifying tools after security systems detected them.
In one of the financially motivated operations they investigated, Anthropic says AI agents did nearly all the work.
Think about that for a minute.
We’ve spent the last few years thinking about AI mainly as something you ask questions or use to help you do a task.
But we’re moving towards something different: you give the AI an objective, it figures out the steps, uses different tools, executes the work and comes back when it needs you.
Obviously Anthropic’s report is showing the dangerous side of that.
But while reading it, I kept thinking about the other side.
If an agent can coordinate a complicated cyber operation, what happens when you give the same level of autonomy to normal business operations?
Instead of opening your accounting software and checking 10 different reports, imagine telling it:
“Close August and show me anything that needs my approval.”
Or:
“Go through our overdue invoices, figure out who needs following up, prepare everything and bring me the important ones.”
That, to me, is where business software is heading.
Not adding a chatbot to every SaaS product and calling it AI.
The software itself starts doing the work.
There’s a catch though.
The more access we give these agents — email, bank transactions, customer records, accounting, files, payments — the more dangerous bad permissions become.
So I don’t think the race is simply going to be about who builds the most autonomous agent.
It’ll also be about who can make that autonomy safe: proper permissions, limits, audit trails and human approval where it actually matters.
This Anthropic report is about cybersecurity, but I think it accidentally gives us a pretty good preview of where software in general is going.
We’re moving from software we operate to software that operates for us.
That’s a much bigger change than just putting AI inside existing apps.