Augmented Materiality Laboratory

Online lectures on platforms like Zoom are widely used all around the world since the COVID-19 pandemic. Though a few studies showed that the teaching quality is relatively good, empirical observation has shown that students get distracted much easier than during face-to-face lectures – and a fortiori compared to a one-on-one teaching session. Previous studies indicate that virtual online environments (including virtual spaces and the use of avatars) can partially compensate for the sense of social presence lost in video conference online lectures. Leveraging on these observations as well as personal experience gathered in the past two years, we propose an original and scalable strategy readily applicable to video-online teaching scenarios (and beyond), bringing back some of the advantages found in a one-on-one personal teaching experience, even when the teacher is imparting a lesson to a large, non-collocated class.

More concretely, we are applying for funding to support the development of a bidirectional semi-autonomous avatar simulating a one-on-one teaching and learning experience. The use of avatars and virtual environments provides the equivalent of a classroom: a neutral and abstract space dedicated to the teaching and learning experience, separated from the private sphere. For these reasons, it has been gaining momentum in online teaching platforms [1], and hailed as capable of eliminating distraction, promoting engagement, and encouraging freedom of expression, among other positive traits. Their principle is simple: each participant (including the teacher) is provided with one’s own (tunable) avatar in a virtualized classroom. While this strategy simulates the dedicated space of a classroom, it does so in detriment of the capacity of personal assessment from the side of the teacher. The problem is exacerbated by the fact that online teaching and learning platforms allow for an arbitrary large number of students.

Our proposal is geared toward solving this problem without restricting the number of students, since this can be seen as an advantage of this technology. The proposed system extends the use of avatars as follows. As with other systems, students see a single teacher avatar, through which the main teacher imparts the lesson. However, when appropriate, control can be passed to a teaching assistant to attend to a single student request, while the others continue listening to the main avatar teacher. Similarly, the teacher(s) will see a unique student avatar representing the whole cohort of students. This single student avatar splits automatically as a student “raise a hand” online in order to ask questions. This gives the choice to the teacher to pass control of the avatar as seen by that student only to a teacher assistant (or other teachers, see below), or to address the request in front of everybody. Since the illusion of a one-on-one teaching experience is both experienced by the teacher and the student, we expect that it will naturally lead to a more personal and natural form of interaction, something greatly missed in the present situation.

The proposed system will be deployed on top of an online teaching platform like Zoom, and the virtual space and  the avatars can be displayed on the computer monitor; however, this research paves the way to the use of more advanced AR technology to insert 3D avatars (in particular that of the teacher) in the real physical space of the student (home, public space), either using a tablet or a head mounted display. In time, the system can be enhanced by introducing semi-automatic behaviours with the help of a context aware AI, enhancing the realism of the one-on-one simulation (choice of gestures and posture, but also character, voice, gender, etc.).

Interestingly, as students and teacher(s) can be anywhere in the world, this strategy may be applied to create a one-on-one personalized massive open online course (MOOC), something that sounds impossible. The key is to understand than in our setup, one can not only recruit a large number of students, but also a (smaller) group of teachers working in concert and puppeteering (and splitting) a unique avatar when required. In the same vein, one can conceive a physical representation of the (single) class avatar, for instance a commercial telepresence robot that could be transported to special locations, such as museums or other locations (including overseas) inaccessible to many. This concept, proposed by the PI when working for the Uruguayan government under the PLAN CEIBAL[2] initiative, was underway before he moved to Hong Kong – the telepresence robot was conceived as a school bus that could be sent to the country’s capital or another country for that matter, and attended and “walked by” a local teacher or guide.