VR and the Brain
An exploration of neuroscience, perception, and immersion
9.S52 (UG) and 9.S915 (G)
6 units (3-1-2), Spring 2024
Wednesdays, 2:00pm to 5:00pm (studio), plus
Tuesdays, 7pm to 8pm (lab)
First session is Wed Feb 7 at 2pm
Instructors: Seth Riskin, Pawan Sinha, Ken Zolot
Teaching Assistant: Aubrey Simonson
plus visiting guests
Virtual Reality (VR) has seen major advances in recent years. Headsets are engineered to couple with the visual brain and communicate along all channels of visual information. In this special subject, we will explore how VR “speaks” with the brain’s early visual processing and consider applications to perceptual learning, low-vision augmentation, and other therapeutic uses.
We’ll convene a project-based studio, to foster collaboration among students of engineering, neuroscience, visual arts/design, and other fields. We'll combine analog light/perception studies (demos and illusions) with VR experiments and experiences to examine and evaluate VR as a tool for targeted stimulation of early perceptual processing. Key questions: can stimuli designed and delivered through headsets strengthen neural pathways for improved acuity, stereopsis, color and contrast sensitivity? Can VR be used as a teaching and learning tool, raising user awareness of perceptual processes? Can simulation/headset be leveraged for communication between low-vision and fully-sighted experiences? The semester will conclude with a project showcase.
Readings:
- Perception Viewed as an Inverse Problem by Zygmunt Pizlo, “Vision Research” volume 41 issue 24, November 2001.
- Stabalized Images on the Retina by Roy Pritchard, Scientific American Vol. 204, No. 6, June 1961
- Lightness Perception and Lightness Illusions by Edward Adelson, chapter 24 in M. Gazzaniga, ed., The New Cognitive Neurosciences, 2nd ed., 2000
- Multiple Worlds by Jan Koenderink, De Clootcrans Press, 2012
- The Potential of Virtual Reality for Inducing Neuroplasticity in Children with Amblyopia, Coco-Martin et al, Journal of Ophthalmology Volume 2020
- Dream Engineering: Simulating Worlds Through Sensory Stimulation, Carr et al, Consciousness and Cognition, August 2020
- Deep Reality: An Underwater VR Experience to Promote Relaxation by Unconscious HR, EDA, and Brain Activity Biofeedback, Amores et al, CHI 2019
- A Framework Toward “Liminal Interactions”, Amores, chapter 2 of PhD dissertation 2020
- Promoting relaxation using virtual reality, olfactory interfaces and wearable EEG, Amores et al, IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks, 2018
- Role of Learning in Three-Dimensional Form Perception, Sinha and Poggio, Nature 1996
- Galea: A Physiological Sensing System for Behavioral Research in Virtual Environments, Bernal et al, IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Christchurch, New Zealand, 2022
- Virtual Reality for Vision Science, Hibbard 2023