Our new paper lead by
@vedanglad w/
@AToliasLab "Letting the neural code speak..."
arxiv.org/abs/2605.12485
We show how to get *monkey* visual neurons to TELL us in *human* language what images make them fire. We do this is an automated verifiable way at scale! How?
1) Build a digital twin of monkey visual areas that can accurately map visual inputs to neural activity.
2) Perform in-silico experiments on this twin to find many complex images that make a model neuron fire.
3) Use a vision-language model to describe these complex images.
4) Verification: use a language-conditioned diffusion model to generate new images, and check they make the monkey digital twin neurons fire a lot.
To our knowledge, for the first time, we have a way to convert monkey vision to human language, and from *human* language to sample infinitely many images that make any given *monkey* visual neuron fire, all in an algorithmic fashion.
For more exciting details, see
@vedanglad's excellent thread!