Opus 5.5 is one of the few models we have seen gain recall and precision together as reasoning effort rises.
Of eight model families we ran at more than one effort setting on dfbench, six traded precision for recall. Opus 5 lost both. Opus 5.5 gained 6.8 points of recall and 4.3 points of precision.
We received early access to @claudeai Opus 5.5 and evaluated its defensive security capabilities on @depthfirstlabs dfbench.
At high reasoning effort, the model achieved 54.9% recall and 48.5% precision, up from 48.1% and 44.2% at medium. Detection cost averaged $8.42 per task, roughly one-third the cost of GPT 5.6 Sol at high and half the cost of Grok 4.7 at high, though with lower recall than both.
Surprisingly, increasing reasoning effort improved both recall and precision. We’ve typically seen a tradeoff in our evaluations, with models finding more vulnerabilities at the expense of more false positives. Opus 5.5 improved on both.
It also reached 79.5% macro recall on differential analysis, up from 75.3% at medium. At $1.78 per task, it showed strong performance in tracking vulnerabilities across code changes.
Sep 23, 2026 · 1:02 AM UTC
5
4
51
5,121
We observed that the reason Opus 5.5 gains recall and precision is that with more effort it searches the same files, but deeper, checking what it might have missed.
For example, in the bug finding tasks below, with more reasoning, it finds the bug by checking what it missed.
1
1
247
Opus 5.5 is also much cheaper than Opus 5, because it uses fewer turns and fewer tokens to find a bug: 36 LLM turns per task against 71, and a third of the output tokens.
It reads larger sections of a codebase at once and decides whether they are relevant. Opus 5 reads smaller sections, so it needs more turns to cover the same ground.
2
3
199
On the flip side, because Opus 5.5 stops earlier, it finds fewer vulnerabilities. Against GPT 5.6 Sol it gives up 4.7 points of detection recall, but has higher precision.
It is stopping earlier because it believes it finds all the vulnerabilities. Its final turn usually says so, in lines like "everything checks out, so I'm ready to write the JSON output" and "I'll conclude there are no findings and return an empty list".
1
1
112
Opus 5.5 is a strong model for vulnerability discovery and validation. It is token efficient, and what it reports is more likely to be real.
However, the coverage may not be what you want. GPT 5.6 Sol, for example, finds more real vulnerabilities.
A combination, using each model for what it is best at, is likely the strongest agentic scanner a security team can use today.
2
78
