In this work, we introduce All-Topographic Neural Networks, which, when trained on visual data, capture topographic features of the visual system and human visual behaviour. With @lu_zejin, @__init_self, Daniel Kaiser, Radek Cichy and @TimKietzmann. 2/10

Background: CNNs are the most commonly used networks to model vision, and have been successful at predicting primate neural activity across multiple hierarchical levels and at accounting for complex visual behaviour. 3/10

One limitation of CNNs in neuroscience is their reliance on weight sharing, i.e., CNNs detect identical features across visual space. This renders them unable to model fundamental aspects of biological vision, such as the origin of topography and its relation to behaviour. 4/10

We overcome this limitation with end-to-end topographic networks (All-TNNs) in which: 1) each unit has its own local RF, 2) units in each layer are arranged on a 2D “cortical sheet” without weight sharing, and 3) feature selectivity varies smoothly across space. 5/10

Upon training, topographical features of the ventral stream emerge in All-TNNs (but not CNNs), including smooth orientation selectivity maps and cortical magnification in the first layer, and category-based selectivity clusters for tools, scenes, and faces in the last layer. 6/10

To study the impact of topography on behaviour, we conducted a human experiment to quantify object recognition performance across space. This yields category-dependent spatial accuracy maps for humans. 7/10

There is category-specific structure in these behavioural accuracy maps. On a variety of metrics, we find that all-TNNs better reproduce this behavioural structure than CNNs. 8/10

Finally, we show that the success of All-TNNs in reproducing human spatial biases in behaviour is directly linked to their topography. 9/10