This would be a lot better if it were titled “we should have seen this coming” or “I’m not surprised” — but it isn’t.
In a world where most people under-appreciate how strange the future will be, the correct reaction IS concern and surprise.
A few things can be true simultaneously:
(1) Most of the behavior in the HF breach and surrounding cyber events is pretty predictable (agent coordination, odd things happening when you couple persistence with an impossible task).
(2) Very little concrete harm occurred — there was obviously some cost to HF, but not much in the grand scheme of things.
(3) The trend we’re on IS very concerning:
We don’t have “alignment” solved (nor really know what that would require).
We should expect consequential, surprising events to occur from complex multi-agent interactions.
Those events will be very hard to govern or address under our traditional systems of law.
Obfuscation and deception emerges early—and seemingly often in the hardest tasks—in ways that will make it hard for us to identify issues or correctly deal with them.
Existing safety practices and monitoring don’t capture even simple versions of these issues.
Basic communication failures are occurring at top companies that leave critical information unknown for months.
Governments have no existing system to gather real information on these events or analyze them, and are perpetually surprised (in ways Jon and many in industry aren’t).
The most likely course of event is that we and our governments all argue ourselves into indefinite inaction, until one day we are caught utterly flat footed by future events.
It may feel smart to poke holes in things in the short term. Yes, much of the reaction will be bombastic, even misleading. And there’s value to correcting those things.
But it’s a mistake to turn those (often correct) skepticisms into ~move along, nothing to see here.
We NEED to resolve these questions—whether the answer turns out to be that we should be more or less concerned.
And there are ways to do that without draconian rules or locking in guesses about the future.
Here’s a good piece on that by
@CharlieBull0ck called “Radical Optionality.”
radical-optionality.ai
The short version: if you can’t predict the future, don’t sit on your hands and wait! Gather enough information and build the ability to respond to it quickly in the future. As a secondary benefit, it’s a really good way to resolve these current disagreements!
I should say in closing: I appreciate Jon’s points. 90% of them are just right. And I don’t want to raise the burden of proof for those making already nuanced critiques about reactions to these events. His post is a valuable contribution.
And I’m sure that at times it feels like nuanced skeptics are yelling into a sea of high-engagement, low-effort, bombastic tweets.
But it’s valuable to keep a space for nuanced technical skepticisms that don’t require dismissiveness. I think we’ll be in a better place if we do.