Re-reading Kaplan et al. and I had forgotten how many banger graphs they had.
1
1
30
2,543
The real lede was how physics-y the vibe was. @ch402 (back in 2021 (!!)) has already noted this, but it's especially striking now given how much of the scaling curve has now played out. colah.github.io/notes/bio-an…
1
10
695
Have you seen the connection between scaling laws and induction heads? transformer-circuits.pub/202…
3
1
15
561
I mention becasue the other part of Kaplan Fig 7 was important in driving intuition for our work
3
11
1,244
Replying to @jacobrintamaki
Looking at loss as a function of token index (as seen here) was very much influenced

Nov 10, 2025 · 3:57 AM UTC

1
15
1,528
Sort replies: Relevant Recent Liked
Replying to @ch402 @jacobrintamaki
Love this—Kaplan’s Fig 7 intuition lines up with induction heads emerging at scale: token‑index loss profiles make the phase transition visible from pattern matching to in‑context generalization.
39