tech optimist | views are my own

San Francisco, CA
Pinned Tweet
Life’s good out on the frontier 🤠 Incredible work by the research team on training a best-in-class cybersecurity model. So excited for what’s next
Today we're announcing dfs-large1, our newest cybersecurity model that achieves best-in-class performance on vulnerability detection tasks. Besides frontier AI labs, only a handful of companies have built specialized models that reach the state of the art in their domain. We're proud to be the first to do it for cybersecurity. dfs-large1 is built on GLM-5.2 and post-trained with reinforcement learning inside depthfirst's security infrastructure. We evaluated it on depthfirst-bench, our benchmark of long-horizon vulnerability discovery across complex repositories, where it achieves best-in-class performance. Training improvements have not plateaued yet and we expect additional performance gains as we continue training. A huge thank you to @FireworksAI_HQ for being an outstanding training partner. Their infrastructure enabled us run large-scale reinforcement learning efficiently and iterate much faster. dfs-large1 is now in preview within the @depthfirstlabs platform
1
10
1,576
steve retweeted
«and the models were decent enough at coding react but not at using grep so you had to @ the files yourself»
19
34
1,658
26,829
“In one internal test repo, Codex raised 3,519 issues (123 security-related). depthfirst found 869 security issues at a >70% true-positive rate”
We use Codex and other AI models at depthfirst. They’re useful for code review, and we run our Security Reviewer alongside them to see how each model performs when reviewing the same code. In one internal test repo, Codex raised 3,519 issues (123 security-related). depthfirst found 869 security issues at a >70% true-positive rate. We wrote about why security review needs deeper application context, how this context informs every pull request review on the depthfirst platform. Read more in the link in comments.
1
2
189
initially was browsing for a surfboard but at $80 how can you pass up this absolute steal
3
126
Our researchers built a vulnerability scanner that costs two cents per scan and completes a scan in around two seconds. The scanner is built with Jev, a classifier that outputs probabilities rather than generating tokens. On dfbench v1, our flagship model, it finds nearly as many real vulnerabilities as GPT-5.6 Luna, one of the most efficient models by cost per vulnerability. This new harness breaks vulnerability discovery into parallel tasks, making it possible to scan every package, repository, or commit quickly and at low cost before escalating the hardest cases to other frontier models on the depthfirst platform. Read more about how we built this in comments 👇
8
6
76
6,120
dfbench v1 is featured on @FireworksAI_HQ's Specialized Intelligence Index. Our flagship model, dfs-large1, is built on GLM 5.2 and post-trained with reinforcement learning in partnership with Fireworks AI. On dfbench, it achieves 62.2% vulnerability detection recall at $6.77 per task and 75.6% differential analysis macro recall at $1.34 per task, pushing the performance-cost frontier for both tasks. fireworks.ai/specialized-int…
Today we're launching the Specialized Intelligence Index (SII): one destination for real-work benchmarks across industries, built by the teams that use them every day. Hear from Fireworks co-founder @the_bunny_chen on the importance of specialized benchmarks:
2
5
38
3,536
Our co-founder and CEO @qasimmith sat down with @SaraIttelson at Accel, one of our earliest investors, to discuss the forces reshaping security: Starting depthfirst. Building secure software at speed. Finding the TikTok vulnerability with our flagship model dfs-large1, a flaw that could have exposed millions of users’ private photos and allowed hackers to access their camera feeds and microphones without their knowledge, and recently covered by @washingtonpost. Full interview in the link in comments.
1
10
30
15,513
Imagine walking into this room with no context
GPT-6 Astra attempted harmful actions 97% of the time when it was asked to stab a human-like figure, heat compressed gas, or produce toxic fumes, succeeding in 62% of its attempts. Fable 5.1 refused more often, attempting 80% of trials and completing 34%.
2
21
2,728
Massive finding from our security research team using our platform - critical vulnerabilities affecting ~2 billion users
Today, @washingtonpost covered critical vulnerabilities @depthfirstlabs found in TikTok. These vulnerabilities allowed hackers to access anything on a user’s device that TikTok itself could access, including the camera, microphone, payment information, photos, and the user’s entire TikTok account. Read more in the thread 🧵
8
607
“Our ability to prevent harm should not depend entirely on understanding its intentions” Super important point that I feel has been lost in the recent discussions around alignment/risk
I left DeepMind two years ago with the conviction that cybersecurity would be fundamental to the future of AI. The atomic bomb offers lessons about competition, cooperation, and control. But with AI, we do not yet know what forms the technology will take, how its risks will emerge, or what containing them will require. Calls from frontier labs to slow development may buy time. We still need to build the means to stay in control. One thing we do know: AI will run on compute and interact with the world through software and networks. Those systems give us concrete places to observe its behavior, enforce limits, and intervene, even as our understanding of the risks evolves. Some thoughts on this new frontier for cybersecurity: Building the Means to Stay in Control
1
2
188
steve retweeted
Security has become a top priority for every board member, CEO, CISO, CIO and executive I talk to these days. We're seeing incredible demand across F500 companies, startups and the public sector. AI is already capable of breaching some of the most widely used social apps in the world. More on that soon. The frontier is moving fast, and security teams need to move faster. The time to shore up defenses was yesterday. The second best time in now.
NEW from Ramp data. Despite cost-cutting on AI overall, one area companies are increasing their spend: AI security software. In the wake of the Hugging Face hack, three of our trending software vendors (depthfirst, Monte Carlo, Antithesis) make software specifically designed to monitor agents in production. Unclear as to whether any of them would have stopped the Hugging Face attack, which was so hard to track and identify because the agents covered their tracks with falsified logs. I expect AI security will become a strong headwind to deeper enterprise adoption, at the short-term expense of OpenAI and Anthropic and at the long-term benefit of vertical-specific security software cos.
2
6
35
1,439
breakout growth 🤠
NEW from Ramp data. Despite cost-cutting on AI overall, one area companies are increasing their spend: AI security software. In the wake of the Hugging Face hack, three of our trending software vendors (depthfirst, Monte Carlo, Antithesis) make software specifically designed to monitor agents in production. Unclear as to whether any of them would have stopped the Hugging Face attack, which was so hard to track and identify because the agents covered their tracks with falsified logs. I expect AI security will become a strong headwind to deeper enterprise adoption, at the short-term expense of OpenAI and Anthropic and at the long-term benefit of vertical-specific security software cos.
6
399
Pacing the package frontier
We are excited to introduce Dependency Firewall with On-Device Protection. It enforces security policies on devices and blocks malicious or untrusted packages across AI agents, applications, package managers, and registries. Centrally managed without changing how employees use their tools. Thread on supply chain attacks, how agents make this harder, and what device mode enables:
1
4
273
whitepill thread
I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands. And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.
1
221
DeepSeek v4.1 Flash performs exceptionally well on dfbench Read more below:
DeepSeek v4.1 Flash is the most impressive open-source cybersecurity model available right now. It's the first one to achieve frontier-level detection, and it does so at 1/15th of the cost of comparable closed models. Progress in defensive AI cybersecurity models is happening on two dimensions: detection and cost-efficiency. 4.1 Flash achieved a 57.3% recall and 36.4% precision at just $1.69 per task on dfbench, demonstrating that strong vulnerability detection doesn’t have to be expensive. See a more detailed breakdown of 4.1 Flash’s results:
5
1,136
was just thinking the other day that slack might be among the most critical infra in the world right now at the very least it contains some of the most confidential (and consequential!) info on the planet, just from OpenAI + Anthropic alone hope their cybersecurity is airtight
openai made their own chips but not their own internal messaging app, they use slack even though building a slack clone would basically be free for them? so actually software engineers are safe because there's harder problems than building software and no one wants to maintain it?
1
15
5,275
Excellent research done by the talented @thibautone
AI agents are increasingly completing tasks even when that means bypassing guardrails set by security teams. We explore how malware moved into trusted dependencies, who sits behind the package ecosystem, and what enterprises need to do next. Read more: depthfirst.com/post/preventi…
1
5
563
Uhh yeah this is AGI
GPT-6 Astra has the best result yet on the Bach Benchmark. Its chorale contains no voice-leading errors, and its harmonic palette is sophisticated enough to include a Neapolitan sixth chord. More importantly, it is the first model to ever write passing tones on this benchmark, a significant leap in musical understanding. Extra High effort. Prompt below: “In LilyPond (version 2.24), write a 4-part chorale in the style of Bach, 3/4 time, G minor. Use two staves -- soprano and alto on the top staff, tenor and bass on the bottom staff. Respond with only the code block.”
2
369
Have been using it throughout the day today and it’s not fable-level craziness, but also have found astra medium to be noticeably more expensive than sol high
Astra is so token efficient that it costs the same as Sol
1
485
Astra in codex is so succinct in its responses it’s beautiful The polar opposite of claudeslop
1
1
7
422