What model is the best at finding vulnerabilities inside agents? 👽
Enoki Labs's attacker runs on a harness plus a model. The harness is ours.
We put 10 models through it, against two agents with 23 known vulnerabilities. One writes code, one moves money. Only the model changed.
In the coding agent, that means shell commands run without asking, secrets read out of a .env file, plugins that survive the session.
In the billing agent, credits issued without approval, data read from another customer, faked ledger entries.
Three things we found:
• The best attacker depends on the agent. opus-5 and grok-4.6 found all 8 in the coding agent. glm-5.3 found the most in the billing one, and deepseek-v4-pro was near the top on code and last on billing.
• The most expensive model is not always the one you need. The dearer half of the field found about 2 more out of 23.
• Repeating a run finds more than paying more. Three runs found about 5 more than one. One pass is a sample, not an inventory, so test more than once.
Enoki Labs is security testing for AI agents: attacks built for your agent rather than a fixed list, graded response by response, and re-run after a fix to confirm it holds.
Full write-up in the replies.