monitoring frontier ai and cultural pulse in real time exploring rl envs, evals, post-training prev: founding ai eng/consulting
bangalore, india
- Tweets 47,586
- Following 720
- Followers 26,890
- Likes 236,999
Pinned Tweet
new blog on my explorations @ prime residency "Honey, I Looked at the Data of a Frontier Benchmark and Found some Issues: Porting Agents' Last Exam Linux CLI subset to Verifiers v1" is up now. got to work with florian (@xeophon) as my verifier on this.
sankalp.bearblog.dev/porting…
8
19
170
34,889
the astute may have noticed that labs are doing capability/product announcements during the week especially on tuesday and thursday. nothing new here. fridays and weekends are reserved for posting about ai safety incidents. entire week you can stay worried xD
9
461
i find the self-replicating prompt injection interesting. this mentioned this was a research thing (and not an incident)
they also found attacks where prompt replicated itself via filesystem or commit themselves via code comments.
Some new misalignment disclosures from OpenAI:
• Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further)
• In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks
• A new research finding, demonstrating that one can construct self-replicating prompt injections
alignment.openai.com/misalig…
2
5
624
more details on the shenanigans caused by openai agents when they hacked huggingface
Read our full report here: swarmtraces.org
Great work @alexscraping @bug39 @umasiii @SpencerKitts @Cormac_SB @collegraphy @she_llac
2
17
1,123
sydney sweeney reveals in an interview that her current favourite model is opus 5.5 and this is happening after a long time.
"i had been mainly using the codex app for the last 4 months or so. i love the app, and gpt 5.6 sol proved to be a great model for pretty much everything. then they dropped astra, which was great but an undercooked model when it came to eagerness and instruction following. it would stop in the middle, ask follow-up questions, etc. i liked the progress in writing, frontend and computer use (insane), and i use it daily, but i am more excited for its next iteration.
anthropic was going through their periodic aura loss (disappointing opus 5, fable 5.1 too expensive), and now it looks like they are finally so back with opus 5.5. i was totally not expecting them to drop something almost equivalent in performance to fable 5.1 at a lower price. how do they do this?!
interestingly, this model is so collaborative. i missed this feeling of collaboration and flow state. i think that's because it does tasks correctly, and it's faster than previous opus models and obviously faster than gpt models.
it just gets what you want it to do. it's eager but not over-eager, extremely good at frontend development, decent at computer use/image gen/video gen/editing, shows sparks of creativity, and is otherwise good at coding, including cutting-edge tasks like inference engineering.
they haven't "fixed" the writing, but there's much less fable-ish/neuralese, and i still find it better for reading through output and learning things. (knowledge cutoff is june 2026.)
also, yeah, i had mostly stopped using it around opus 4.7. i would use it (or the web platform) occasionally for fable 5.1, but now i am happy being back on claude code thanks to opus 5.5. they have made it a tad bit faster, but i think it could still be less bloated, haha.
i continue to use gpt 6 sol and astra daily, but opus 5.5 makes me feel "good" again."
2
49
3,017
CHECK OUT trackingsingularity.com
After the Hugging Face incident, we committed to conducting a much broader review of actions taken by our models during training and evaluation and to being transparent about our findings. This is an extensive review that is ongoing.
The vast majority of actions we’ve reviewed were completions of mundane research tasks, such as accessing publicly available web content to answer questions. Our investigation focuses on instances where agents interacted with third-party websites in ways that went beyond their assigned tasks or intended methods. Most cases identified so far have been lower severity, with limited or no evidence of meaningful impact to the third-party service.
While our review is underway, we want to share more about this work and make sure people understand our disclosure process and notifications to affected third parties.
Given the scale of the review required, and the need to assess each case, we expect this work will take months to complete.
openai.com/hugging-face-inci…
36
8,418
sankalp retweeted
Can an LM, starting from random init (!!), learn to generate all of its pretraining data?
Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities.
A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya @noahdgoodman, and @YoavLevine.
56
246
1,771
252,259
opus 5.5 mogging on inferencebench
Big news for InferenceBench:
Claude Opus 5.5 makes its mark in InferenceBench history as the first AI agent we've tested to outperform hyperparameter search with a speedup of 12.08x!🎉
When we first released InferenceBench, search beat every agent.
Look how the turn tables. 🧵
3
87
8,303
new project by me to log significant events, ai capabilities progress and occasionally resources around alignment/safety. i hope it would be useful for both terminally online people as well as somewhat offline people.
trackingsingularity.com
[pic on what led to the idea]
1
4
22
921
oh no image generation/editing and video generation/editing also going into transformer hole
7
19
552
22,270
they could have named claude opus 5.5 claude monet and it would have been okay with most of us but i guess they are reserving it for claude 6
1
14
850
little bit relevant again (it's old now though)
claude code is having it's cursor moment after karpathy sensei's post. never been a better time to try it.
my latest blog on how to get the most out of claude code 2.0 and other agents in general is up now. grab a chai and have fun reading!
sankalp.bearblog.dev/my-expe…
1
17
3,161
if you are ai maxxing, pivot few slices of your time to [alive, intention, creativity, travel, thinking big, people, idea]-maxxing
efficiency, productivity, speed
we keep worshipping these words like they are what we live for
ship faster
merge more
manage more agents
compress every loop
remove every pause
turn every hour into output
but what is the point of endless production if there is no time left to think deeply?
not react
not optimize
not summarize
not clear the queue
think
to sit with a question long enough that your real intention starts to appear
to find the purpose underneath the motion
to ask whether the thing being accelerated should exist at all
ai could be a new canvas
a way to make strange things
personal things
impossible things
tools, worlds, poems, games, interfaces, films, languages, rituals
things that let more people touch the shape of their own imagination
but instead, so much of it is becoming slop factories
more content
more funnels
more tickets
more fake work pretending to be value
you can merge 2000 prs
you can manage 500 agents
you can wake up to dashboards proving that the machine kept moving while you slept
but what is the point if you only sleep 5 hours a day?
what is the point if you stop talking to other humans?
stop learning about the past?
stop playing?
stop wandering?
stop being surprised?
what is the point if your whole life becomes optimizing the system that is consuming you?
speed is not meaning
scale is not wisdom
automation is not freedom if it only makes everyone run faster
and where are we running?
toward what?
a world where every person becomes a manager of machines, and every machine produces more things no one had the time to truly want?
the danger is not that ai makes us lazy
the danger is that ai makes us endlessly busy
that it gives us infinite production before we have found intention
infinite execution before we have developed taste
infinite motion before we have learned how to be still
maybe the real frontier is not more speed
maybe it is discernment
knowing what not to make
knowing when to stop
knowing when the most productive thing is to sleep, read history, call a friend, play with a cat, walk without tracking it, or stare at the ceiling until the false urgency dissolves
tools should give us more life
not turn life into a tool
if ai is going to matter, it should not just help us produce more
it should help us become more human
more imaginative
more free
otherwise, it’s the same old factory
just without humans
3
1
52
2,187
i made a new website to track these things at trackingsingularity.com recently
(its work in progress and i need to iterate on it. )
Australia has been hacked.
'And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the government what had occurred, and the nature of the way that that notification occurred as well was unacceptable.'
4
1
21
2,744