Exploring the representational geometry of object representation in the ventral stream using brain-behavior correlations

Cohen, Konkle, Nakayama & Alvarez

The visual processing stream contains regions that selectively respond to different objects, and the representational geometry within each region can be measured by the similarity of responses across different items (Kriegeskorte et al., 2008). Here we asked if interference between objects in a perceptual task is predicted by the representational geometry across different regions of the visual system.

To measure perceptual interference, sixteen participants performed a visual search task with eight categories: faces/bodies/buildings/cats/cars/chairs hammers/phones. On target present trials, one target was shown amongst seven distractors from one category (e.g. one face/seven chairs), and reaction time was measured for each target-distractor pairing. Reaction times were used as an index of perceptual similarity between categories, yielding an 8x8 behavioral-similarity matrix.

To measure the representational geometry of different visual areas, six new participants were scanned using fMRI while viewing individual items from each category. The neural patterns for each category were correlated with one another across several cortical regions, yielding an 8x8 neural-similarity matrix for each region. Do these neural-similarity measures correlate with the behavioral-similarity measures? We found strong correlations across the ventral/dorsal pathways (ventral temporal, r=0.79; lateral temporal, r=0.62; occipitoparietal, r=0.54), but not V1-V3 (r¬=0.13). To test the uniformity of this representational geometry, lateral temporal, ventral temporal, occipitoparietal, and V1-V3 were divided into ten ROIs based on overall voxel activity. Surprisingly, the correlations in each sub-region remained high throughout ventral- and lateral-temporal cortex (P<0.05 in each sub-region). Furthermore, we found significant correlations within both FFA (r=0.66) and PPA (r=0.66).

These results suggest that the representational geometry in higher-level visual areas constrains object perception and is highly uniform across ventral visual cortex. The uniformity of this representational geometry, despite differences in response selectivity, suggests that different regions of ventral visual cortex extract, and make explicit, different subsets of highly correlated perceptual features.

Cohen, M., Konkle, T., Nakayama, K., & Alvarez, G. A. (2014). Exploring the representational geometry of object representation in the ventral stream using brain-behavior correlations Talk presented at the annual meeting of the Vision Sciences Society, May 16-21, St. Pete Beach, FL.