Augmented Materiality Laboratory

Abstract

This research project aims to investigate the extent to which augmented reality (AR) experiences can elicit measurable physiological, psychological, and neurological responses in the user’s body.

To this end, we have developed an AR experience that allows users to see and hear their own hands burning while looking through a video see-through head-mounted display (VST-HMD)
To date, we have conducted a series of three experiments. The first experiment examined the prevalence of heat illusions and the physiological and psychological stress responses triggered by the AR experience. Half of the participants reported a heat sensation on the affected left hand. All participants experienced a significant increase in skin conductance during the experiment. In addition, participants who experienced a heat sensation had a higher skin conductance response.The second experiment examined whether a heat illusion resulted in thermoregulatory responses. Using a Laser Doppler Flowmeter, we demonstrated that skin blood flow in the affected hand changed significantly in some participants.
Experiment three examined whether this AR experience could influence participants’ perception of pain. The virtual flames on participants’ hands led to a decrease in the temperature at which they perceived heat-related pain.

The main conclusions of the first three experiments can be summarized as follows: (1) AR can reliably induce cross-modal illusions in subjects through visual and auditory stimuli. (2) Involuntary heat illusions in AR can lead to some thermoregulatory responses, such as an increase in blood flow in the skin. (3) AR experiences may strongly affect thermal pain perception.
Our research will continue with several planned follow-up experiments to further explore how AR experiences can have measurable effects on the user’s body and mental state. This will provide deeper insights into the perceptual and cognitive effects of AR experiences. The results of our experiments could be relevant in a neuroscience or medical context.

Collaborators

  • Colin Blakemore, Department of Neuroscience, City University of Hong Kong, Hong Kong
  • Jan Schnupp, Department of Neuroscience, City University of Hong Kong, Hong Kong
  • Ryszard Auksztulewicz, European Neuroscience Institute, Goettingen, Germany
  • Lai Ling Gladys Cheing, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong

Publications

[1] D. Eckhoff, A. Cassinelli, and C. Sandor, “Heat Pain Threshold Modulation By Experiencing Burning Hands in Augmented Reality,” in 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct. 2021, p. 2. doi: DOI 10.1109/ISMAR-Adjunct54149.2021.00079. PDF

[2] D. Eckhoff, “[DC] Psychophysical Effects of Augmented Reality Experiences,” in Proceedings – 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops, United States, Mar. 2021, pp. 739–740. doi: 10.1109/VRW52623.2021.00252. PDF

[3] D. Eckhoff, C. Li-Tsang, G. Cheing, A. Cassinelli, and C. Sandor, “Investigation of Microcirculatory Effects of Experiencing Burning Hands in Augmented Reality,” in Proceedings – 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops, United States, Mar. 2021, pp. 569–570. doi: 10.1109/VRW52623.2021.00167. PDF

[4] D. Eckhoff, A. Cassinelli, T. Liu, and C. Sandor, “Psychophysical Effects of Experiencing Burning Hands in Augmented Reality,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Cham, 2020, vol. 12499 LNCS, pp. 83–95. doi: 10.1007/978-3-030-62655-6_5. PDF

[5] D. Eckhoff, A. Cassinelli, and C. Sandor, “Exploring Perceptual and Cognitive Effects of Extreme Augmented Reality Experiences.” PDF