Summary

This Virtual Reality (VR) module was developed to teach students in large lecture classes critical thinking skills. To move away from the large lecture format, the module was developed to be deployed in the college library and get students to engage in an immersive context where they are asked to assess the validity of source material and statements and sort them to build an argument about topics they have heard of in lecture.

The hypothesis is that movement reinforces the learning of critical thinking skills rather than just the content of the lectures, which are more passive for the student. In order to build this module, faculty and instructional partners build prototypes and tested each on campus. The module is based on an information framework that is easily adaptable to different topics and readings so as to make the module reusable and customizable.

The VR module had multiple prototypes, which were all known as “Che’s Village.” The module prototypes were interactive VR experiences in which students made decisions and sort a variety of pieces of information based on a 1959 speech by Che Guevara that was used in a large lecture class on Latin American history.[1] The text was also used in a small face-to-face seminar with 20 students, who were also invited to experience the VR module. This happened in the pre-pandemic era, when students at the University of California, Riverside (UCR) could take some online courses but were still taking most of their classes on campus, when staff were on campus, and sharing headsets was not the hazard it might seem to be in the current COVID-19 environment.[2]

Instructional Partners

The VR modules discussed below were deployed at UCR’s Rivera Library, the Humanities and Social Science library. They were set up in a large room with three or more headsets, and students signed up for a time slot over the course of a week. This was done in collaboration with the Library which was considering developing a digital makerspace with a VR component.

Audience

This is a college-level VR experience for undergraduate students. The instructor in this case teaches in the History department, but the framework is designed to be adaptable to any content.

Student at large public research institutions are subjected to the inherent inequity that a while their instructors are all expected to and rewarded for their research, students rarely have the opportunity to learn how to do research in lower division classes. With this in mind, and especially when teaching about inequality, poverty, social unrest, and change that marked the 1960s in Latin America and globally, a motivating idea behind this VR module was to challenge some of the built-in inequities in many parts of the higher education system.

Curricular Context

The exercises detailed below describe the development of prototypes of a Virtual Reality instructional module. The exercise in VR was designed as a class in which students would learn something in VR, not to teach students how to code or create mixed reality experiences. As such, the text contextualizes the pedagogical and technological context in which we developed a teaching module that included VR experiences.

Faculty observed students in the interactions with the hardware and the VR experience to improve on the different prototypes. The lessons from these experiences are valid across teaching platforms, but in these instances they were deployed in multiple courses, including large online courses with up to 400 students.

Learning Outcomes

1. To organize content by source and bias.

2. To sort through relevance of source depending on question that is being answered.

3. To expose students to content in immersive context.

4. To introduce students to digital environment.

5. To lead students through critical thinking exercise in immersive environment.

Preparation

Building a set of interactive exercises in VR requires coding and developing skills, which often limits the use of VR to faculty with those skills. This project is based on a VR framework that is easily customizable without sophisticated coding skills or training.

The VR module as such exists in its prototype form, multiple phases of prototyping are explained below. So far, there are four prototypes of the VR experience connected to historical material about Che Guevara and the Cuban Revolution. The first two iterations were prototypes of educational VR that tested the usability of the concept itself. The third prototype emphasizes critical thinking about source material and is designed to invite students to assess and sort different types of historical information against a variety of hypotheses. The fourth focuses exclusively on UX and usability of the movement in VR without concerns for content.

Technical Requirements

VR headsets loaded with framework - last prototype files here:

Because the framework is content agnostic, this highlights the physicality of the UX for the students and the information loading experience for faculty/developer.

Type of experience: immersive, critical

Materials

The VR module is designed to fit into an introductory Latin American history class. Before experiencing the module, students had lectures on the context of the Cuban revolution, the Cold War, and the politics and economics of the late 1950s and early 1960s, and they had read the assigned speech in which Che Guevara presents the case for land reform in the aftermath of the overthrow of the US-backed Batista regime and a return to prosperity for the island without US involvement.

The choice of text is not neutral – this is a speech Che Guevara made on the radio in the early days of the revolution – Havana had just been reached; the dictator Fulgencio Batista had barely left the island. Che’s speech is not a political tract – it lays down the fundamentals of the economic system the revolution is engaging with and vows to change. Che invokes legacies of Spanish colonialism as well as the consequences of American imperialism as the reasons the radical shift needs to happen – but his proposals are rooted in a Marxist materialist vision prioritizing everyone’s access to the means of production. It is a very useful text with which to introduce students to this moment when post-colonialism is articulated and to present them the ideals of the revolution as Che Guevara presents them and ask them to assess these ideals in relation to modern day Cuba and Latin America.

Session Outline

Prototype 1: Can Students Be Immersed in Content?

The first prototype really focuses on content and context – it creates a context inspired by reality. In the first version of this VR build, the student entered the experience and saw a desk, where they could look at a copy of the speech by Che Guevara that had been assigned in class.[3] The environment was a relatively realistic jungle setting corresponding to the Sierra Maestra in Cuba, with less realistic images that acted as portals into themes in the speech.

The VR module placed students at a desk at an encampment – the environment was rich with a feeling of place, it included soundscapes and parrots that occasionally flapped their wings and flew by.

The floating images (see above) were odd- and this led to a new landing screen. It was adjusted to three banners representing the lines of critical inquiry that students could point and click, each line of inquiry opened a portal to layers of information in multimedia formats – there were video clips, sound clips, images, and text.

There was significant reticence to add too much text since reading text in VR remains a relatively unpleasant exercise. The experience was not a reenactment of a chapter in Che Guevara’s life - it was an immersive research experience about this chapter in Cuba’s revolution, and it was an invitation to explore the speech and to visualize new and unexpected connections in the source material.

Historical Source Material

The framework was loaded with a selection of publicly available sources. These were integrated them into the module. The table below summarizes the sources we used in VR module.

The first VR build was immersive and allowed users to explore 360 degrees of the surrounding environment from a seated position, but it had limited kinesthetic opportunities. The build was tested for usability and user experience and even in these early prototypes, the outcomes were encouraging.

This first build was tested with 80+ users in an HTC Vive VR headset in early February 2017. Based on exit questionnaires and time spent in the environment, users were able to draw conclusions based on the data and cite their reasoning after only a few minutes of exposure to the material in VR. It was estimated users would spend 5 to 15 minutes in the experience, and most student users stayed in the experience for close to 15 minutes and sometimes more than that.

One clear result of the early tests that the experience generated a period of undivided focus for the user—the immersive experience was in fact immersive. The environment in this prototype is rich in detail and is not connected to the web or to a smartphone, allowing users a rare moment of distraction-free immersion that is very hard to replicate outside of VR. The group of students who did the VR experience[4] also demonstrated a better ability to address the research question on their final exam. This is not by any means a scientific research protocol, for which we are seeking funding, but it is a positive outcome.

Prototype 2:  From content to body movement (kinetic) to increase learning in an immersive experience

This sketch illustrates the greatest value of the VR module Building based on the first prototype, and how students spent time in a distraction free environment.

The first prototype really focuses on ways to show content, but showing content in VR is not the best use of the functionality of VR. Students interact with content in a myriad of ways constantly: the course material itself is content, the lecture is content, their phones are full of content.

A second prototype was tested with a similar number of users in early June 2017. This build pivots the articulation of the VR framework’s usability. The value of the VR experience was to have students engaged without distractions, and this was leverages for maximum learning outcome.

Prototype 3: How much movement is too much movement?

The third prototype was tested in March 2019. These iterations of the build more fully leveraged the space that VR offers and required movement with controllers from one side of the space to the others. The focus was less on the multimedia aspect of VR—the desk and the text, as well as the visual representation of the sources and video were eliminated —and the space was reorganized to maximize movement and physical experience. The movement in VR was intended to mirror that of moving between sources and evidence when doing research in a library.

This prototype was developed in Unity for the Windows Mixed Reality headset, which provides handheld motion controllers allowing navigation and interaction.

Users were asked to stand during the experience, but the exercise was also possible in a seated position. Users enter a low-poly natural environment modeled after a generic jungle. The space is expansive, users can move around the virtual space as much as their physical tether permits, and greater distances can be covered via teleportation using the motion controllers.

In this build the historical sources we had been using were represented as a physical object, a “data block” that users could picked up and examine. These blocks received skins so that they look sort of like books. The intent was to visually reinforce the objectives of the exercise, namely that what the users are sorting and evaluating are historical sources, which will often be contained in books. The working hypothesis was that users might be able to be provoked into internalizing the connection between the physical form and the information it contains.[5]

Data Blocks in Motion

Each data block/book in the VR experience is mapped to a quotation/content from one of the sources related to the topic of the Cuban revolution. The exercise is designed to model agency within the information science structure without making it explicit that that is what the user is doing.

In this build, the experience asks students to physically turn a piece of evidence around to analyze how it sheds light on a particular issue. The intent was that as students manipulated the data blocks by turning them around, they would internalize the cognitive message that to access knowledge from any evidence, it needs to be looked at from multiple sides.

Students/users were then led through a series of exercises, and they were asked to sort the different data blocks in terms of topic, source, bias, and role in constructing an argument for a specific hypothesis.

Each exercise built on the previous one, exposing relevant information along the way. For example, at the start of the first exercise, the data blocks contain only the quote of a historical source. The user is asked to sort it according to primary topic, choosing from a) US-Cuba relations, b) land reform & peasant life c) economy. Following the completion of this exercise, the blocks are annotated with icons indicating their topics, and subsequent blocks added to the environment arrive with icons already in place. In this manner, the user gradually builds an understanding of the data and is then able to use it to form higher order arguments.

As for the ability to retain the lessons of the exercise, a random identifier was assigned to each student who volunteered to test the build. The random identifier was an animal name, which was shared with each student. Four weeks after the VR experience, students were asked to answer an anonymous quiz that asked if they had participated in the VR test and if so, what animal they had been assigned. Most students who did the VR experience remembered their assigned animal. This is hardly a proxy for critical thinking but serves to indicate that there is a connection between the experience and a specific memory of it.

Prototype 4: Leveraging Functionality for Realism

The user outcomes from the last round of testing pointed to some obvious needs for improvement. One of the things any VR teaching module will need to invest time in is that there must be a simpler way for users to move about the space. Teleportation needs to be seamless, and in this specific case, a mechanism that allows users to hold on to the books/data blocks  while teleporting.

Whereas in a realistic setting, one would need to hold the books to have the books, in VR this requires the user to “hold” the book with the trigger of the controllers. While realistic in essence, it isn’t how human actually hold books and in VR it is also very impractical because most users will drop the book as they try to teleport.

One of the key lessons from the development from the first prototype to this one is that immersivity does not require realism. In fact, the need to recreate reality can obscure the point of the exercise. In this module, the need for students to hold on to a book without holding on to the handheld controls is a priority, so departing from reality is inevitable.

To address this relationship between realism and response, a separate VR experience was designed that nothing at all to do with Che Guevara’s speech or any content from class.  This experience focused exclusively on the user experience with different interaction modes. Different users were given different ways to grab and hold blocks, to measure which modalities worked best. To do this, upon entering the VR experience, users were presented with an animal.

Depending on the animal and the set-up, some users sorted the series of questions about their animal by moving blocks.

Other users chose answers by pressing buttons on a screen.

This last iteration of the VR build was in development until early 2020 and focused on integrating the lessons learned from the user experience tests into the Che content framework. The teleportation problem was well on its way to be solved, but the pandemic put the project on hold. When the VR module is back in rotation it will be possible to assess see the user experience issues have been solved to focus exclusively on the cognitive lessons in VR.

Assessment

This goal of the VR module is to do something that is generally hard to do in large lecture classes and bring some of the more targeted instructional opportunities of small classrooms to a large lecture class. The VR module overturns some of the constraints built-in to traditional large lecture classes that make it difficult for students to learn by doing. One can lecture on history and on the texts and documents that make up the historians’ archive but lecturing on research is very different from doing it practically. This VR experience puts students in an environment in which they can make decisions about the themes in the speech, explore additional information from a variety of sources, and choose which ones were better to answer specific questions about the reading. The test of efficacy happens outside of VR, in assignments they do that demonstrate their skills and understanding of the material.

The VR experience was set up in the UCR Rivera Library over a sequence of three days –students could sign up or drop by the spend some time in the module. The VR module provide students with an immersive space of doing. The intention with this VR module is not to create a reenactment of the speech nor to immerse student into Havana in 1959.

The possibility of using immersive experiences to push students to think critically came out of a critical assessment of the limits of a research game. This game “Digital Zombies” focuses on information literacy and leads students through any library. Digital Zombies was and continues to be a successful information literacy game.

Practical skill-building modules in VR were the inspiration to move beyond the Digital Zombies game and find a way to teach critical thinking skills in VR.[6] Research suggests that cognitive skills are heightened and complemented by physical movement – and the VR modules were developed to test if movement in a virtual space could have a similar cognitive effect.[7]

The VR module is explicit in its need to co-exist in a hybrid learning environment, where there are multiple bridges between digital and analog structures of knowledge. The successive versions of the VR prototypes are all reliant on digital, virtual, and traditional modalities of instruction coexisting and reinforcing each other.

Ultimately, it remains to be proven that a student who experiences part of their learning in any VR module become a better thinkers outside of it, but the prototypes outcomes detailed below, are encouraging.

Reflection

March 2020 and the ensuing pandemic put any type of campus testing on hold, and there is a very real concern that sharing any type of headset will be highly unlikely in the future. VR is still the preferred mechanism for embodied experiential learning, but sharing tools and headsets is unrealistic right now, and it is uncertain when it might be safe to get large numbers of students in a library to participate in the project.[8]

Should there be a moment when sharing headsets and doing large scale testing of learning again, it will be done with a solid research protocol. To demonstrate the value of VR as a pedagogical tool that might serve to provide large lecture classes with a means to “learn-by-doing” and experiential learning even, or perhaps especially, at scale will require the right research protocol.

It has always been our desire to organize a research protocol to measure the relationship between the immersive nature of VR and the cognitive process in it. A key issue with VR is how to assess the extent to which participants are interacting with and attending to the content and how this contributes to learning outcomes. Collecting eye-tracking data informs us of how participants attend to the material. Eye-tracking would allow us to measure whether elements are seen and the extent to which interaction with material is aimless or guided. Comparing learning outcomes provides critical information on how different strategies relate to different learning outcomes and the extent to which attention to each piece of material contributes learning. Assessing the relationship between what the user looks at and what the user does and connecting that to other qualitative outcomes outside of VR gives us with insight not only about how VR can help educate in the humanities, but how we want to design a humanities VR experience for maximum effectiveness.

Doing this however requires a very different funding source and research protocol that we do not have, so it remains an idea in search of a project partner (and an idea in expectation of a safer health environment).

For now, the prototypes and what we have learned from them allow us to imagine that a VR learning module has the potential to return the student to a traditional method of learning, to that small classroom feel, not by recreating the classroom in VR, but by conjuring up the learning mechanism of that small classroom that has largely disappeared in the large lecture class. A goal of the 21^st century classroom should be to involve students actively in the process of meaning and knowledge construction no matter what the size of the classroom or the number of classmates.

The small seminar teaching environment is increasingly beyond the reach of large public universities, but immersive technologies make it possible to imagine a world in which many students can explore and experience scholarship in a context that privileges intellectual experience and cognition over content delivery. We hope that there will be possibilities to explore this safely in the future.

Application in other courses

When planning to build a historical and instructional VR module, keep track of the source and origin of any content included in the module. The developers keep track of the 3D assets and the code, but on the instructional side, the list of contents is important not only for record-keeping purposes, but it became an incredibly important piece to keep track of as we adapted the VR module.

The initial content list will become a key element of the planning stage for any VR module. As our intention was to build a sorting process in VR that replicated some of the cognitive processes involved in research, we needed to have a clear idea of what was being sorted and how it could fit into a template. We coded for a series of variables that made sense considering our source material, for example, chronology, source bias, relevance to specific questions, content themes like economics, demographic, sociology, or culture. Different source material will require different variables and themes, but if these are systematically organized, the template will work for most any content.

Part of what makes VR such a rich learning experience is that it offers the possibility of interactivity beyond a point and click. There is a kinetic element that really helps tie a cognitive moment to a physical one. If you are trying to teach students something in VR that they will need demonstrate outside of VR, consider the kinetic elements of that lesson.

Even once the conceptual framework is solid, you may still find that you need the correct VR mechanism to provide the correct kinetic signal. The sorting mechanism needs to be supported by the available technology in terms of hardware and coding. The only way to get that balance right, namely between the available VR technology and the effect you want it to generate in the conceptual framework, is to prototype, prototype, prototype.

Each new iteration will reveal the limits of the solutions you figured out to the last problems. Even in a relatively simple environment, the complexity of what this framework is trying to do, namely to get students to sort conceptual ideas about a reading, requires we iron out the mechanics in the build, which only become obvious as we test it and test it. In this sense, building an interactive experience in VR requires multiple prototypes and trials at every stage of the experience.

Additional Materials