#12: Enhance virtual reality

All text taken from "Grand Engineering Challenges of the 21st Century", National Academy of Engineering

To most people, virtual reality consists mainly of clever illusions for enhancing computer video games or thickening the plot of science fiction films. Depictions of virtual reality in Hollywood movies range from the crude video-viewing contraption of 1983’s “Brainstorm” to the entire virtual universe known as “The Matrix.”

But within many specialized fields, from psychiatry to education, virtual reality is becoming a powerful new tool for training practitioners and treating patients, in addition to its growing use in various forms of entertainment. Virtual reality is already being used in industrial design, for example. Engineers are creating entire cars and airplanes “virtually” in order to test design principles, ergonomics, safety schemes, access for maintenance, and more.

What is virtual reality?

Basically, virtual reality is simply an illusory environment, engineered to give users the impression of being somewhere other than where they are. As you sit safely in your home, virtual reality can transport you to a football game, a rock concert, a submarine exploring the depths of the ocean, or a space station orbiting Jupiter. It allows the user to ride a camel around the Great Pyramids, fly jets, or perform brain surgery.

True virtual reality does more than merely depict scenes of such activities — it creates an illusion of actually being there. Piloting a Boeing 777 with a laptop flight simulator, after all, does not really convey a sense of zooming across the continent 5 miles above the surface of a planet. Virtual reality, though, attempts to re-create the actual experience, combining vision, sound, touch, and feelings of motion engineered to give the brain a realistic set of sensations.

And it works. Studies show that people immersed in a virtual reality scene at the edge of a cliff, for instance, respond realistically — the heart rate rises and the brain resists commands to step over the edge. There are significant social applications as well. It has been shown that people also respond realistically in interactions with life-sized virtual characters, for example exhibiting anxiety when asked to cause pain to a virtual character, even though the user knows it’s not a real person and such anxiety makes no rational sense. It is clearly possible to trick the brain into reacting as though an illusory environment were real.

What are the practical applications of virtual reality?

Virtual reality offers a large array of potential uses. Already it has been enlisted to treat people suffering from certain phobias. Exposing people who are afraid of heights to virtual cliff edges has been shown to reduce that fear, in a manner much safer than walking along real cliffs. Similar success has been achieved treating fear of spiders.

Other experiments have tested virtual reality’s use in treating social anxieties, such as fear of public speaking, and show that it can be a successful treatment for some more serious disorders, such as post-traumatic stress disorder. Virtual reality also offers advantages for various sorts of research, education, and training. Some neuroscientists believe that virtual reality experiments can provide insight into the nature of awareness and consciousness itself. Surgeons can practice virtual operations before cutting into real people; soldiers can learn combat tactics in virtual worlds without shooting real bullets.

Virtual reality could also be used in business, advancing video conferencing to a level in which people located in widely dispersed parts of the world can interact in a shared environment and carry out tasks together. Meeting the engineering challenge of allowing dispersed people to seamlessly see, hear, and touch each other, as well as share real objects and equipment, would be particularly useful for the military and emergency response teams, too.

All of these scenarios involve outfitting the user with a virtual reality interface — often a display screen mounted on the head so as to cover the eyes and ears — that communicates with a computer. The computer stores all the necessary information to generate the virtual scenes and sounds. Typically, the visual and auditory information is transmitted separately to each eye and ear, giving realistic stereovision images and two- or three-dimensional impression of sounds. As users move their heads, the computer quickly generates new images to reflect what people moving about in a real world would see next. Since head movements result in corresponding changes to what is seen, as they do in real life, this acts as a very powerful mechanism for immersing the user in the virtual world.

What engineering advances are needed?

For virtual reality systems to fully simulate reality effectively, several engineering hurdles must be overcome. The resolution of the video display must be high enough, with fast enough refresh and update rates, for scenes to look like and change like they do in real life. The field of view must be wide enough and the lighting and shadows must be realistic enough to maintain the illusion of a real scene. And for serious simulations, reproducing sensations of sound, touch, and motion are especially critical.

While advances have been made on all of these fronts, virtual reality still falls short of some of its more ambitious depictions. Fine-grained details of the virtual environment are impossible to reproduce precisely. In particular, placing realistic “virtual people” in the scene to interact with the user poses a formidable challenge.

“Rendering of a virtual human that can purposefully interact with a real person — for example, through speech recognition, the generation of meaningful sentences, facial expression, emotion, skin color and tone, and muscle and joint movements — is still beyond the capabilities of real-time computer graphics and artificial intelligence,” write neuroscientist Maria V. Sanchez-Vives and computer scientist Mel Slater. [Sanchez-Vives, p. 335]

Yet virtual reality users routinely respond to even crude “virtual people” as though they are real. So one of the challenges of virtual reality research is identifying just what level of detail is necessary for a user to accept the illusion, in other words to respond to virtual events and simulations in a realistic way. Already, it seems that visually precise detail may not be as important as accurate reproduction of sound and touch.

Touch poses an especially formidable challenge. For some uses, gloves containing sensors can record the movements of a user’s hand and provide tactile feedback, but somewhat crudely. That’s not good enough to train a surgeon who, when cutting through virtual tissue, should feel different degrees of resistance to the motion of a scalpel at different places along the tissue. Moreover, with today’s technology you can’t feel an accidental bump against a virtual piece of furniture.

Efforts to solve such problems are in the beginning stages. One possible approach would make use of electrorheological fluids, which alter their thickness when exposed to electric fields of different strengths. Perhaps an advanced virtual reality computer could make use of this effect to send electrical signals to adjust a glove or garment’s resistance to touch, providing touch feedback to the user.

It may not be virtual reality per se, but a related concept also seems to be growing in cyberspace, as the World Wide Web has become host to whole worlds populated by virtual people guided by their real-world owners. One such site, known as Second Life, already has millions of participants, some who just visit, some who buy virtual property, establish virtual businesses, and communicate and form relationships with other inhabitants through various communication channels.

What’s more, such worlds could be in the process of merging with the real world, as computer records of the physical environment (as available via Google Earth or Microsoft’s Virtual Earth) could be interlaced with the sites like Second Life. It would then be possible to virtually visit real locations, explore a city’s restaurants and hotels, and engage in other virtual tourist activities.

References

Doug A. Bowman and Ryan P. McMahan, “Virtual Reality: How Much Immersion Is Enough?” Computer 40 (July 2007). http://doi.ieeecomputersociety.org/10.1109/MC.2007.257

D. Klein et al., “Modelling the response of a tactile array using electrorheological fluids,” J. Phys. D: Appl. Phys. 37 (2004), pp. 794-803.

Wade Roush, “Second Earth,” Technology Review (July/August 2007).

Maria V. Sanchez-Vives and Mel Slater, “From presence to consciousness through virtual reality,” Nature Reviews Neuroscience 6 (April 2005), pp. 332-339.