security assessment manager, engineer @sigp_io
- Tweets 1,235
- Following 446
- Followers 9,901
- Likes 4,658
Pinned Tweet
Found a live bug in EigenLayer (this was a few days before the cantina contest). It was discovered during reviewing the offchain sidecar rewards calculation.
6
14
162
16,323
took a long time to go through the interview process but it was well worth it, we ended up taking on 6 interns which will be split up into 2 cohorts over the next several months
We will be taking on 3-4 security interns this round. 6-8 weeks paid internship. I will be acting as one of the mentors
Apply here
5
47
5,159
ERC4626 with low nSLOC, so plenty of existing findings to pattern match with, AI has better chance to perform well this type of codebase
High rate of false positives ranging from 30%-100%. Therefore AI doesn't replace security engineers as the work becomes judging whether the findings are valid or not.
Let's say in 2 years AI can find 90% of the bugs on a complex 3000 nSLOC codebase. The false positive rate is still likely to be high, so a large part of our work becomes more like a judge in audit contests - assessing impact and giving recommendations
5
3
67
7,076
Nice list!
Though this list is intended for dev positions. So I am wondering if people think security folks should also be expected to know all this, or at least the easy questions?
Because sometimes SRs learn a new language on the fly during an audit, and rely on their research skills/threat modelling to find bugs, despite not knowing the specific language well at all.
We classified very similar questions as "easy" here: rareskills.io/post/solidity-…
2
1
33
6,150
We have wrapped up the first round interviews
The interview had a short technical component which caught some candidates off guard, though I was surprised that some had a hard time despite their impressive backgrounds.
Do you think these questions are fair game for every SR to know?
- what is integer overflow
- if you were using an earlier version of solidity that didn't revert on overflow, what check would you implement to prevent it
- what is the difference between external/public functions, pure/view
- explain reentrancy
- explain tx.origin/msg.sender
- explain call/delegatecall
The int overflow and follow up on how to prevent it tripped a lot of people up.
We have manually reviewed all the applications and will be sending out 20 interview invites soon.
To give an idea of the quality, the people who have made the cut have had 50+ H/M bugs in audit contests, multiple top finishes, private audit portfolio.
35
2
114
30,859
verify the output
AI PROMPTING → AI VERIFYING
AI prompting scales, because prompting is just typing.
But AI verifying doesn’t scale, because verifying AI output involves much more than just typing.
Sometimes you can verify by eye, which is why AI is great for frontend, images, and video. But for anything subtle, you need to read the code or text deeply — and that means knowing the topic well enough to correct the AI.
Researchers are well aware of this, which is why there’s so much work on evals and hallucination.
However, the concept of verification as the bottleneck for AI users is under-discussed. Yes, you can try formal verification, or critic models where one AI checks another, or other techniques. But to even be aware of the issue as a first class problem is half the battle.
For users: AI verifying is as important as AI prompting.
3
1,717
Auditing the response from AI is part of the workflow, it also has the benefit of solidifying your own understanding of the code.
For example if there is a complex function:
- Let AI explain it.
- Verify if the answer is correct.
- If there was any hallucination, explain the correct answer to AI
- Ask follow up questions again recursively.
As you do this the context window of the chat improves so that the AI becomes better at answering questions about this codebase.
Your understanding of the codebase also grows so you can more quickly pick up any hallucinations
4
5
52
4,290
We have manually reviewed all the applications and will be sending out 20 interview invites soon.
To give an idea of the quality, the people who have made the cut have had 50+ H/M bugs in audit contests, multiple top finishes, private audit portfolio.
13
2
92
45,512
We will be taking on 3-4 security interns this round. 6-8 weeks paid internship. I will be acting as one of the mentors
Apply here
Sigma Prime is hiring 🚀
We’ve just added 3 roles to our GitHub
- Blockchain Security Intern
- Rust Engineer
- DevOps Engineer
Help shape the future of web3 with us 👇
14
4
176
37,659
planning out a more structured internship intake - security engineers and an internal LLM role, will post more details when it gets finalized
4
48
3,850
A negotiated white harbour agreement post hack, at best might serve some purpose of protecting you from a civil case, but the protocol has no power deciding whether you get charged criminally by the state, that agreement between you and the protocol have no sway here.
It is the state vs you in a criminal case, not protocol vs you. The protocol has no power to protect you from prosecution even if they wanted to.
5
21
2,493