Red Team Update
In the final hours before
@callebtc resumes his role as chief clank officer and wartime general of red team, I wanted to give some kind of update about today.
We are accelerating.
Not only in the expanse of repositories covered (over 300) and our spend (almost $40k), but in the efficiency to quickly identify where parts of the codebases will break.
Seeing dozens vulnerability reports in an hour will do that to you.
Another fascinating thing is how different members of the team are able to get entirely different outcomes based on our own applied knowledge.
An example was when I scanned a repo and found some middling concerns. I showed the results to another member of the team, and with 1-2 sentences and maybe a block of code, they were able to upgrade to high or even critical vulnerabilities.
That happened twice.
Its almost like the agent can smell out something is wrong, but hasn't been trained enough yet to clearly verbalize it, so the slightest nudge of good knowledge/context will send it all on another recursive flurry deeper.
I think there is some deeper lesson there about AI's productivity gains further accelerates the division of labor, but I'm too tired to get so abstract.
The Red Team harness I mentioned yesterday was quickly thrown together over 20-30 minutes laying out conceptually, who I someone who has no security research, would think this would work.
I now have multiple responses to disclosures telling me it was very well written, but I'm wondering if the bar is so low because they think it'll be llm slop. Maybe because its written in a way an LLM can pick right up and give exact feedback.
Upon receiving a disclosure, one developer critically read through the whole report and gave a lot of very strong constructive feedback (you know who you are, thank you).
I asked if they could give it as a lesson to my clanker, to which they returned to me a 50 kb markdown file. After which, I interviewed it for 30 minutes, integrating behaviors I wanted to focus on into the harness behaviors into the model, and quickly found even stronger success as a result.
Seeing how impactful that feedback session was, I'm making a point tomorrow to NOT be scanning repos (as much).
My focus primarily will be working on the harness and systems itself. I have a lot of ideas, maybe I'll come back tomorrow with lessons learned.
Outside the harness, I want to focus more on operations. Every step of the process is chaotic right now. I accidentally nuked the repo this morning by not paying attention to Calle's new style guide and frantically had to unwind it.
Another delay was at one point I walked away from my computer at the wrong time and the balance on
@opencode spiked down to -$1000. It killed dozens of ongoing agents/subagents, and got frantic pings about scans going down.
20 minutes later, I noticed that none of my credit cards worked anymore. I then got the error "try debit card", and that failed too!
I watched the balance tick to under $100, knowing I was close to stalling everyones jobs again when suddenly it worked (thx
@jayair for quick support it worked out).
I also want to focus on triaging all of our disclosures. We should be able to turn these around a lot faster now, and cleanly sort what has and has not been shared yet.
In other news...
To be blunt, I'm still numb over the death of the cold card.
I have not heard of many stories about the lives ruined, and that is probably on purpose because I've made
@AnchorWatch and the Red Teaming my sole focus.
I have blocked out the thought of even starting to process the catastrophic harm that has been caused to my friends and their families.
There is a tragic irony that is not lost on me that good people wanting to store their wealth, their time, their future dreams in a more honest and fair system were exploited by the thing they took refuge in.
I truly believe these events have been a spiritual attack on bitcoin, and I mean that in the literal senses of the words.
While things are not easy right now. I have the highest conviction ever in my life that the idea and technology of Bitcoin is worth fighting for.
To that end. There is no Bitcoin without self custody. This is non negotiable.
That does not mean everyone must self custody, but the option to is necessary for a free and prosperous society which respects property rights.
I think one of the biggest lessons from all of this is one of risk management.
There was a latent Risk of Ruin in how many people held their bitcoin, and unfortunately what many thought to be impossible, came to pass.
I will find time to grieve, but today isn't the day.