Coding benchmarks did not match what we were seeing when using LLMs for formal verification. So we built our own. Today we’re releasing Ethereum Verification Benchmark V2

Along the way, we found something even more interesting. With enough effort, smaller models replicated 96% of the tasks solved by frontier models. The scaling-law sidecar remains a useful view of how inference effort changes proof solve rate.

In our formal audits, we start by mapping the full system dependencies to then formally verify their smart contract. For @unlink_xyz, this gave strong confidence in their contract plus a clear view of what their system depends on and needs monitoring going forward.

We've been banned by @OpenAI three times to benchmark GPT 5.6 Sol against the most popular Solidity Audit Skills 1. GPT + @cyfrin Skills - 16/35 -vulnerabilities found 2. Raw GPT 5.6 Sol - 12/35 vulnerabilities found 3. GPT + @pashov Skills - 12/35 vulnerabilities found 4. GPT + @QuillAudits_AI Skills - 11.3/35 vulnerabilities found Our Conclusions 1. Models alone are good enough for general vulnerability detection. A generic audit skill didn't help (exception for @cyfrin). We feel skills work best when encoding your own context (architecture, invariants, threat model) 2. Permissionless security work is getting harder. We got banned three times before being verified for OpenAI Cyber Preview access. 3. Do we actually still need auditors? Cyber is a market: attacks improve, defense adapts, and vice versa. AI may not do an auditor's job well yet, but eventually it will (even for formal verification). The question is: how confident are you that it's good enough, and how well do you understand it? Given blockchain's security stakes, we believe that's worth the time and effort to verify.