Convergence and divergence in the neural organization
of object responses to pictures and words
Konkle, Wang, Peelen, Caramazza, Bi
Visual information is transformed from early sensory formats into increasingly
abstract representations of its content. We probed this abstraction by
exploring the convergence in the neural responses to pictures of objects and
their spoken names, taking a broad look at several major semantic divisions
between object categories. Our aim was to explore which neural regions
show reliable responses that are specifi c to visual pictures, specifi c to auditory
words, or common between these two modalities, using a data-driven
clustering approach. Using fMRI, we measured neural response patterns
to objects from 18 broad semantic categories, presented as both pictures
and auditory words, in 16 participants. We used a clustering technique
to group together voxels by their response profi le similarity over these
18 categories, in a way that is agnostic to where the voxels are located
and whether they refl ect responses from the visual or auditory modality.
They key advantage of this procedure is that it simultaneously discovers
common and unique structure across both modalities without presupposing
any regional boundaries in advance. This analysis identifi ed several
regions with similar neural profi les to pictures and words (parahippocampal,
transverse occipital sulcus, retrosplenial cortex), which primarily had
a response preference for inanimate categories of objects. In contrast, other
neural regions were only reliably modulated by pictorial stimuli (lateral
occipital, fusiform), and these regions primarily had a response preference
to pictures of animate entities. Taken together, these results demonstrate
a surprising and currently unexplained link between the neural organization
of broad object domains and activations by different modalities.
Konkle, T., Wang, X., Peelen, M., Caramazza, A., Bi, Y.(2015).
Convergence and divergence in the neural organization of object responses to pictures and words.
Talk presented at the annual meeting of the Vision Sciences Society, May 15-20, St. Pete Beach, FL.