Strategies underlying face and object recognition

In a series of studies, we have investigated the nature of information used in ecologically important tasks of object recognition. Of primary interest to us is the recognition of faces when image quality is degraded due to refractive errors, amblyopia or large viewing distances. We have estimated lower-bounds on the image resolution needed for face detection and identification, and titrated the contribution of cues such as pigmentation and configuration. The resulting data have not only provided clues regarding the nature of representations the brain uses for achieving its robust performance, but have also served as starting points for the creation of robust artificial vision systems.

Sinha, P., Balas, B. J., Ostrovsky, Y. and Russell, R. (In Press). Face recognition by humans. In Face Recognition: Advanced Modeling and Methods, Academic Press.

Russell, R., Sinha, P., Biederman, I., & Nederhouser, M. (In Press). Is pigmentation important for face recognition? Evidence from contrast negation. Perception.

Riesenhuber, M., Jarudi, I., Gilad, S. and Sinha, P. (2004). Face processing in humans is compatible with a simple shape-based model of vision.. Proceedings of the Royal Society of London, B. (Suppl.), 04BL0061.S1-S3.

Sinha, P. (2003). The use of 2-D similarity metrics for 3-D object recognition. IETE Journal of Research, Vol. 49, Nos. 2 and 3, pp 113-125.

Yip, A. and Sinha, P. (2002). Role of color in face recognition. Perception, Vol. 31, pp 995-1003

Sinha, P. (2002). Recognizing complex patterns. Nature Neuroscience, Vol. 5 (suppl.), pp 1093-1097.

Sinha, P. & Poggio, T. (1996). I think I know that face..., Nature, Vol. 384, No. 6608, pp. 404.

Vocabulary of recognition

We have complemented our experimental investigations of the nature of information used for various recognition tasks, with theoretical proposals for how such information can be formally represented. We have developed two related representation schemes that encode images in such a way as to attain a significant measure of tolerance to changes caused by illumination variations, resolution reduction and noise. The approaches are biologically plausible and, in computational simulations, are found to improve recognition performance over that obtained with conventional strategies such as those involving Gabor wavelets. Furthermore, the schemes suggest interesting new ways of conceptualizing the transforms effected by early visual areas in the mammalian brain, and have begun guiding neuro-physiological investigations of novels kinds of receptive-field structures in the primate visual pathway.

Balas, B. J. and Sinha, P. (In Press). Receptive field structures for recognition. Neural Computation.

Sadr, J., Mukherjee, S., Thoresz, K. & Sinha, P. (2002). The Fidelity of Local Ordinal Encoding. In T. Dietterich, S. Becker & Z. Ghahramani (Eds.), Advances in Neural Information Processing Systems 14. MIT Press: Cambridge, MA.

Sinha, P. (2002). Qualitative representations for recognition. In Lecture Notes in Computer Science, Springer-Verlag, LNCS 2525, pp 249-262.

Oren. M., Papageorgiou, C., Sinha, P., Osuna, E. and Poggio, T. (1997) Pedestrian detection using wavelet templates. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico.

Neural correlates of recognition

In conjunction with our behavioral studies of face perception with degraded images, we have attempted to determine the neural correlates of this task. Using fMRI, we have investigated neuronal responses when human observers view clear and highly degraded images of faces. We find that a specific brain region (in the fusiform gyrus) is activated when participants view clear facial images. This is not surprising, and several past studies have reported this result. What is surprising, however, is that we find that this region is also activated even when the images are so degraded that the intrinsic facial information (the pattern of eyes, nose and mouth) is entirely obliterated, so long as the surrounding contextual cues (such as the presence of a body) suggest that the degraded region might be a face. In other words, the neural circuitry in the human brain is such as to be able to use context to compensate for extreme levels of image degradations. We are following-up on these studies with single unit recordings in monkey IT cortex (in collaboration with Prof. Earl Miller) to investigate the neural encoding of facial identity in highly degraded images. Besides helping to elucidate the functional architecture of the brain's recognition machinery, these investigations are also informing our work on neurodevelopmental disorders which are characterized by difficulties in information integration.

Cox, D., Meyers, E. and Sinha, P. (2004). Contextually evoked object-specific responses in human visual cortex. Science, Vol. 303, No. 5667, pp 115-117.

Roy, J., Sinha, P. and Miller, E. (2004). Responses of dorsolateral prefrontal cortex neurons in the monkey to blurred and rotated face images. Society for Neuroscience Abstracts.

Vaina, L. M., Solomon, J., Chowdry, S., Sinha, P., Belliveau, J., W., Gross, C. G. (2001). Functional neuroanatomy of biological motion perception in humans. Proceedings of the National Academy of Sciences 98, 11656-11661.

Clinical aspects of recognition

We have developed a technique called 'Random Image Structure Evolution' (RISE) that provides a convenient way of assessing various aspects of high-level visual function. We have used this technique for tasks as diverse as examining neural correlates of object perception, on the one hand, and probing recognition-skill differences between children with normal and abnormal development, on the other.

Sadr, J. & Sinha, P. (2004). Object Recognition and Random Image Structure Evolution . Cognitive Science, Vol. 28, pp 259-287.

Pollak, S. and Sinha, P. (2002). Enhanced perceptual sensitivity for anger among physically abused children. Developmental Psychology, Vol. 38, No. 5, pp 784-791.