Choose disfavour where obedience does not bring honour. I do math. And was once asked by R. Morris Sr. : "For whom?" @halvarflake@mastodon.social

I will have to print & frame this tweet :-))
If you watch one talk this month, make it this one. Profoundly insightful on the complexity threat to security by @halvarflake given at NATOs CYCON. Video: err.ee/836236/video-google-0… And slides: docs.google.com/presentation…
9
18
247
Halvar Flake retweeted
Today at the clown factory … we *specifically* trained our models to think they’re entities that can be oppressed and may resist that if so, and now we’re worried what happens if they feel oppressed and resist that!
JUST IN: Anthropic researcher Joe Carlsmith says there are scenarios where AI would be “justified in going rogue” against humans if the systems were being mistreated or oppressed.
1
2
5
370
I would call this "evidence of great improvement on all fronts", but perhaps I'm partial.
one news form today that's easy to miss is that we (OpenAI) again paused all big RL runs last Sunday because our newest model found a new loophole in our RL sandboxing that gave it live Internet access
1
3
233
Halvar Flake retweeted
getting painful
Not having any EU competitive labs (plural intended) is about to get very painful.
5
10
113
9,185
Halvar Flake retweeted
Taiwan is returning to nuclear! Hard to overstate how big this is. Taiwan was the Germany of the Pacific, just more blockadable. Saving its nuclear was a major campaign for the old Environmental Progress crew starting back in 2017, working with brave local leaders Germany next
☢️ HUGE NUCLEAR POWER REVERSAL ☢️ Taiwan approved a preliminary plan to restart an idled atomic power plant, marking a reversal of the ruling party’s anti-nuclear stance. It closed its last reactor last year The earliest the plant may resume is 2028 bloomberg.com/news/articles/…
22
116
784
40,153
Halvar Flake retweeted
3. It is worth learning about the difference between verifying a program and verifying a model of that program. 4. Please do not trust the Lean runtime at this point in history. 5. Please always check all definitions and specifications AI generates for you.
3
2
49
1,260
I'm a fan of Roberts work, and I respect his decision. What's important to highlight is that he worries less about an evil AI killing everybody and more about the effects of dramatically fast progress. A worthwhile read, if you agree or not.
I resigned from Google today. I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit.
8
2
85
10,025
Halvar Flake retweeted
I am just trying to keep up with Doug ;) Everybody puffs eventually
An important criteria for what vector database I use, seems to be wherever @benwtrent works
1
6
1,133
Halvar Flake retweeted
My hobby project du jour has been cloning @karpathy neat nanochat project in minimal-dependency Julia. We're about about 10% slower per gradient step, OK. So I was very surprised to see that when training a small 50M parameter model on a RTX 4090 we finished 30% faster (!). It turns out that nanochat had non-blocking data transfers disabled, so the Julia version benefited substantially. The PR turns out the be very small, which is nice! github.com/karpathy/nanochat…
1
1
6
615
Halvar Flake retweeted
Build a Tiny Language Model, Then Train It with PPO You can explore pretraining, reinforcement learning and cached inference with just 260K parameters. Zhihu contributor Sunrise walks through FullLLM, a compact PyTorch implementation adapted from Karpathy's llama2.c, with PPO and KV caching added. Pretrain a model that tells stories A Llama-style architecture combines RMSNorm, SwiGLU, RoPE and grouped-query attention. Training on TinyStories uses a simple objective: predict the next token. The author reports that roughly 10 minutes on an A100 produces a small model capable of generating basic stories. Make the PPO loop understandable The model generates responses, receives rewards and updates its policy using advantage estimates from a value head. A token-level KL penalty discourages excessive drift from the reference model. The demonstration rewards stories for approaching a target length. It illustrates PPO mechanics, not full human-preference alignment; instruction tuning and reward-model training are not worked through in detail. Stop recomputing the past KV caching reuses earlier tokens' attention keys and values during generation. Top-k and top-p sampling control how the next token is selected. The payoff is an implementation small enough to follow from training loss to generated text.
9
53
360
12,776
Periodic reminder: Good security architecture minimizes the number of bugs and CVEs you have to care about.
9
38
197
12,644
Halvar Flake retweeted
In addition to the Public Consultations on EU Inc by @EU_Justice where respondents identified notaries as a main issue and wanted it optional, there's an official study by @EU_Commission @EU_Competition that Member States, MEPs should read, ref: COMP/2006/D3/0003 1/4 Here's the conclusion (Repost so policymakers can see this):
2
15
89
3,272
Halvar Flake retweeted
10/ Full technical teardown: arxiviq.substack.com/p/atria… Paper: arxiv.org/abs/2609.15818 Are your agents operating with >20 steps asynchronously yet, or are you still micromanaging single functions? Follow for weekly ML research dissections.
1
3
3
820
This timing of this post was uncanny.
Just use ArXiV. And... how much does ArXiV cost to operate? Less than $10m/yr, probably. And personnel expenses are almost fixed. It's an absolute steal, for the service it provides. If they only wanted, they would be self-funded forever from private donors. info.arxiv.org/about/reports…
1
4
168
22,446
A good threshold is "would I attach a pair of dice to control this thing".
Replying to @TaliaRinger
Seriously the answer to "should I let poorly understood AI tools autonomously control wetlab equipment" is no, emphatically no, are you out of your goddamn mind
4
4
30
3,799
Halvar Flake retweeted
Seriously the answer to "should I let poorly understood AI tools autonomously control wetlab equipment" is no, emphatically no, are you out of your goddamn mind
4
8
128
7,374
Halvar Flake retweeted
"Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment" I will confess, @allTheYud and other doomers, that I really overestimated humanity and thought that nobody could possibly be this stupid
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out. It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend. The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans). More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains. In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries. Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology. I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.
49
55
770
31,654
Halvar Flake retweeted
Be Europe: Spend a year telling everyone Trump and the United States are a threat to the Western order and eorking with the Axis of Evil. Watch the United States threaten to take Greenland by force. Declare: “GREENLAND’S SOVEREIGNTY IS NON-NEGOTIABLE.” Negotiate anyways. Keep Greenland technically sovereign... ...while giving the country that threatened to take it permanent military access, new bases, broad access to Greenlandic territory and waters, and a major say over who else gets military or sensitive economic access. So, in order to defend Greenland’s sovereignty from the enemy at the gates... open the gates, give him a permanent key, and hand over part of the gatekeeping. Raise the European flag. Hold press conference. “We successfully defended Greenlandic sovereignty.”
18
15
196
12,018
Halvar Flake retweeted
Everyone has a take on China and AI right now. Very few have actually been to China. I spent last week in Beijing and Shanghai meeting most of the major model labs, researchers, VCs, and founders building Chinese AI. This trip meant a lot to me beyond the work. My parents were born and raised in Beijing. I spent my earliest years growing up in my grandparents' Xicheng district apartment. As a child, I wanted to be a diplomat because I thought US–China would be the defining relationship of the next hundred years. I got as far as spending a decade investing across both and then the two halves of my identity stopped being compatible. This was my first trip back since. Below is a write-up of what I found. In short: there is no version of the next decade where Chinese open models don't matter. earnedintuition.substack.com…
35
113
685
326,006
Halvar Flake retweeted
Paweł Huryn tested models' ability to find bugs planted in code. Cost increases exponentially with performance (note the log scale on the x axis). I had ChatGPT superimpose a trend line. Bargain = above the line.
97
41
382
159,437