AI engineer at Maze. Sometimes I share machine learning & pop sci visualizations
England
- Tweets 1,645
- Following 274
- Followers 12,394
- Likes 4,779
Pinned Tweet
If you have a distribution, with density p, that you don't know how to sample from, you can still estimate integrals like ∫f(x)p(x)dx by sampling:
Sample from a different distribution, with density q, then weight your samples by p(x) / q(x)
This is called importance sampling
7
120
893
Yet another Opus 5.5 p(doom) music video
I noticed a lot of the other ones I'd seen for this song had the same imagery, and I assumed they were all using one as a reference, but no, the model does just latch onto the same few ideas. I had to specifically prompt *not* to use some of those images
The jump in computer use (and "taste") is really quite impressive. I'm not a fan of AI for art but there's obviously a huge market for it once the slop factor is gone (/ it's not recognisable as AI generated).
30
24
331
40,442
It seems like an obvious way to "pace the frontier" and reduce risk in the short term is just not to run large agent swarms. Unless/until you can trust your automated monitoring, don't run anything with that volume of activity
At the very least, only give a small number of agents in the swarm access to the outside world, maybe throttle their access. Slow down external interaction to the point where you can manually monitor it
Unless swarm size is the only scaling law the frontier labs have, it seems like all you lose by holding off on large multi-agent runs is solving the next millennium prize next year instead of next week
466
Ｂｌｏｇ ｐｏｓｔ ｓｈｏｗｓ
ｈｏｗ ｔｏ ｇｅｎｅｒａｔｅ
ｆｕｌｌｙ ｊｕｓｔｉｆｉｅｄ
ｍｏｎｏｓｐａｃｅｄ ｔｅｘｔ
ｗｉｔｈ ｎｏｎ－ｇｒｅｅｄｙ
ｃｏｎｓｔｒａｉｎｅｄ ＬＬＭ
ｓａｍｐｌｉｎｇ
echostatements.net/posts/202…
by @EchoStatements
1
5
758
It seems (?) like there isn't an easy way to use PyTorch's transformer implementations autoregressively at prediction time efficiently; in each iteration of the loop the whole sequence attends to the whole sequence. Here's a solution I came up with: gist.github.com/grey-area/10…
2
4
2,317
I hope either somebody finds this useful or there's an easier way that somebody can share with me
Explanation of the problem and the solution on the other site. Hope that's legal.
sigmoid.social/@andrewm_webb…
sigmoid.social/@andrewm_webb…
1,333
A fun example of a falsehood spreading through citation: according to (most of) these 47 papers, 19,349,663 is prime
2
1
30
Bonus points to Jan Zacharias for the only publication I could find exposing the 'alleged prime number' as a fraud!
refubium.fu-berlin.de/bitstr…
5
Andrew M. Webb retweeted
Let's ride a Martian space elevator—from Pavonis Mons to areostationary orbit!
The left hand side looks down on the north pole, and the ellipse shows at each moment the free-fall trajectory the elevator car would follow if it detached from the cable
(More details in thread) 1/
2
6
41
Happy Halloween! I decided to get in the spirit and dress as the walking dead
(I've no idea if non-British followers will know who this is)
4
21
Andrew M. Webb retweeted
A simple visual aid to see why the area of a circle is half the circumference times the radius. Based on a section of @stevenstrogatz' new Infinite Powers book
13
289
1,130
If you have a distribution, with density p, that you don't know how to sample from, you can still estimate integrals like ∫f(x)p(x)dx by sampling:
Sample from a different distribution, with density q, then weight your samples by p(x) / q(x)
This is called importance sampling
7
120
893
How efficient this is, in terms of number of samples required, depends on how similar the two distributions are.
You usually want the distribution you sample from to have heavier tails than the target distribution: it's better to oversample the tails than not sample them at all
1
28
If I spend a few hours in a day learning something new and abstract, that night I'll often have non-visual dreams: the 'dream' is just repetitive, nonsensical thoughts. It feels like repeatedly trying out new ideas but on nonsense data.
Anyone else experience this?
4
1
33
It results in not-very-restful sleep and I first experienced this with learning programming, so I still call them 'coding nightmares'
1
5
Another version of this: the two groups move left and right together, but change in size so that the overall mean always moves in the opposite direction
A different take on visualizing Simpson's paradox, which (to me) makes it easier to see what's going on
Each group's histogram moves to the left, but the groups change in relative size so that the overall mean moves to the right 1/5
4
32
Andrew M. Webb retweeted
An animation showing how rockets get to orbit, with details in the thread 1/
(Animation at 10x real time)
9
83
404
A different take on visualizing Simpson's paradox, which (to me) makes it easier to see what's going on
Each group's histogram moves to the left, but the groups change in relative size so that the overall mean moves to the right 1/5
3
52
355
