This project explores how the human brain and body respond to Augmented Reality experiences and how these findings can be used to develop novel medical therapies.
This research project aims to investigate the extent to which augmented reality (AR) experiences can elicit measurable physiological, psychological, and neurological responses in the user’s body.
To this end, we have developed an AR experience that allows users to see and hear their own hands burning while looking through a video see-through head-mounted display (VST-HMD)
To date, we have conducted a series of four experiments.
The first experiment examined the prevalence of heat illusions and the physiological and psychological stress responses triggered by the AR experience. Half of the participants reported a heat sensation on the affected left hand. All participants experienced a significant increase in skin conductance during the experiment. In addition, participants who experienced a heat sensation had a higher skin conductance response.
The second experiment examined whether a heat illusion resulted in thermoregulatory responses. Using a Laser Doppler Flowmeter, we demonstrated that skin blood flow in the affected hand changed significantly in some participants.
Experiment three and four examined whether this AR experience could influence participants’ thermal and thermal pain perception. We found that AR-induced heat-illusions create analgesic as well hyperalgesic effects as participants begin to feel heat related pain at lower temperatures and cold related pain at higher temperatures. The experience also impacts significantly on the lowest temperature at which participants starts perceiving warmth. This opens up the fascinating perspective of being able to use carefully crafted AR experiences to modulate and mitigate clinical pain.
The main conclusions of the first three experiments can be summarized as follows: (1) AR can reliably induce cross-modal illusions in subjects through visual and auditory stimuli. (2) Involuntary heat illusions in AR can lead to some thermoregulatory responses, such as an increase in blood flow in the skin. (3) AR experiences may strongly affect thermal perception and thermal pain perception.
Our research will continue with several planned follow-up experiments to further explore the neural correlates of these illusions. This will provide deeper insights into the perceptual and cognitive effects of AR experiences. The results of our experiments could be relevant in a neuroscience or medical context.
Collaborators
- Colin Blakemore, Department of Neuroscience, City University of Hong Kong, Hong Kong
- Jan Schnupp, Department of Neuroscience, City University of Hong Kong, Hong Kong
- Ryszard Auksztulewicz, European Neuroscience Institute, Goettingen, Germany
- Lai Ling Gladys Cheing, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong
Publications
D. Eckhoff, C. Sandor, G. L. Y. Cheing, J. Schnupp, and A. Cassinelli, “Thermal Pain and Detection Threshold Modulation in Augmented Reality,” Frontiers in Virtual Reality, Aug. 2022. doi: 10.3389/frvir.2022.952637 PDF
D. Eckhoff, A. Cassinelli, and C. Sandor, “Heat Pain Threshold Modulation By Experiencing Burning Hands in Augmented Reality,” in 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct. 2021, p. 2. doi: DOI 10.1109/ISMAR-Adjunct54149.2021.00079. PDF
D. Eckhoff, “[DC] Psychophysical Effects of Augmented Reality Experiences,” in Proceedings – 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops, United States, Mar. 2021, pp. 739–740. doi: 10.1109/VRW52623.2021.00252. PDF
D. Eckhoff, C. Li-Tsang, G. Cheing, A. Cassinelli, and C. Sandor, “Investigation of Microcirculatory Effects of Experiencing Burning Hands in Augmented Reality,” in Proceedings – 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops, United States, Mar. 2021, pp. 569–570. doi: 10.1109/VRW52623.2021.00167. PDF
D. Eckhoff, A. Cassinelli, T. Liu, and C. Sandor, “Psychophysical Effects of Experiencing Burning Hands in Augmented Reality,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Cham, 2020, vol. 12499 LNCS, pp. 83–95. doi: 10.1007/978-3-030-62655-6_5. PDF
D. Eckhoff, A. Cassinelli, and C. Sandor, “Exploring Perceptual and Cognitive Effects of Extreme Augmented Reality Experiences.” PDF