Go to previous article
Go to next article
Return to the 1994 VR Table of Contents
By: Stephen Marcus
Graduate School of Education
University of California, Santa Barbara 93106
Developments in Virtual Reality technology often far outdistance our sense of how best to understand them or best adapt them to teaching and learning. Nevertheless, when we see, for example, David Rokeby's "A Very Nervous System" helping children translate fingertip, eyelid, and face movements into musical sounds, colors, and words, or Vincent John Vincent's Mandala system transforming students into human paint brushes or musical instruments, we can sense the truth of Arthur C. Clarke's dictum that "any sufficiently advanced technology is indistinguishable from magic."
At various laboratories and art installations around the country, and in some malls and video arcades, you can now enter worlds that resemble the ones you're generally familiar with, but in which the laws of the universe are sometimes modified or eliminated. To do this, you sometimes wear special headgear that tracks head movements and provides a three-dimensional image (a kind of Walkman for the eyes). There is also a glove or other device (e.g., airmouse or spaceball) that tracks the position and configuration of your hand. The apparatus is connected to a computer that provides the visual displays and responds to your head and hand movements (and sometimes provides sounds, pressure, and resistance to movement).
These and similar technologies are used to create a "virtual reality," a simulation that you seem to enter physically. It's a kind of "deep medium." Randall Walser, of the Autodesk Research Lab, suggests that if text "tells, and video and film show, then a virtual reality embodies the world it creates." Virtual realities dissolve the line between the interface and the innerface of the worlds you wish to explore.
John Perry Barlow, noting that the only part of himself he saw in a simulated world was "a glowing, golden hand," realized that he didn't "seem to have a location exactly. In this pulsating new landscape, I've been reduced to a point of view. The whole subject of 'me' yawns into a chasm of interesting questions."
In a very different setup, David Rokeby's "A Very Nervous System," your body's movements (with no glove or goggles) are tracked by video cameras and translated into music. Space itself is now the interface between you and the computer; space has become a medium in itself. People who have experienced this sort of environment sometimes note that the relationship between cause and effect seems to disappear. Rob Appleford, for example, noted that, "[As I walked back to the university after spending some time in this system] a truck splashed through a puddle, and I caught myself trying to make that neat noise again by retracing my steps."
Rokeby's system has been used to allow quadriplegics to perform with professional jazz musicians. In one case, the waving of fingertips plays a virtual piano. In another case, a mouth stick moving slightly in the air becomes a percussion instrument, playing virtual drums.
A related kind of system, "Mandala," from Vivid Effects, also has applications for people with special needs. In this case, you are in front of a video camera with a blank background behind you. Your image is transferred into a computer screen that has special images on it (e.g., musical instruments, floating alphabet letters, or a hockey rink). As you move in front of the camera, your image on the screen can interact with the objects there: playing the instruments, grabbing letters to form words, blocking shots. You (or is it your image?) have become an icon, interacting with other icons on the screen.
Many people's thinking includes pictures and sounds as well as words -- unless, like Einstein, you also think in colored patterns and muscle sensations. And consider Temple Grandin: Ph.D., author, world-renowned animal science specialist, and autistic. She constantly "runs" simulations, visualizing the animal from different angles, different distances, zooming in or wide-angle, even from a helicopter view -- or I turn myself into an animal, and feel what it would feel." Grandin was, in fact, "astonished when she discovered that her own near hallucinatory power of visual imagery was not universal" (quotations taken from Oliver Sacks's profile of her in The New Yorker, Dec 6, 1993). Anyone familiar with the look-and-feel of VR glove-and-goggles systems will see a tantalizing similarity with the ways people like Grandin (whether or not identified as autistic) experience and communicate their worlds.
It's hard to communicate, in print, the look and feel of entering virtual worlds. (Can you easily describe to a non-reader what it feels like to be "lost in a good book"?) Having tried, however, to make strange things familiar, it might be instructive at this point to make some familiar things strange.
Consider this. Text simulates thought. It's an artifact, representing with its own particular richness and its own unique limitations the ineffably complex workings of the human mind going about its business of making sense of things. Text is a version of the thought that the text represents. It's a working model of what's on your mind. As with other kinds of simulations, you can interact with and affect text (aka reading and writing). Text is virtual thought.
We've long since become accustomed to a range of materials that embody text, paper being the predominant one (which some people, interestingly, equate with being "carved in stone").
Now consider this. A word processor creates virtual paper. The technology, in fact, fills a new medium with an old content. We tend to think of a word processor as a kind of way-station on the way to a printout, "hard copy." We imagine there's a sort of paper scroll moving past the "window" of the monitor screen. This simulated paper has, however, powers and abilities far beyond those of normal paper. The "videotext" sometimes blinks, ripples, and slides. It can disappear and reappear. It can change its shape and sometimes its color. It can have embedded in it "buttons" that will play sounds and recorded voices or show moving pictures. This sort of virtual paper is less and less designed to be "printed out" (although more and more we're hearing the phrase, "print to video"). Increasingly, this sort of document's home is the computer environment in which it was created.
There are, in addition, examples of "virtual books" that are currently available commercially. The illustrated children's stories in the Discis collection, on CD ROM discs, will read themselves to you in English or Spanish, in word or phrase groupings of your choice. They will provide you with definitions of each word and will remember the words you didn't know. You can have the books instantly "rewritten" in different typographies. The recently released Expanded Books, from the Voyager company, provide moving images and sounds to accompany the text, along with special text-searching and annotating features. The "books" are stored on computer disk, for use with Apple Computer's PowerBook series of computers.
These two examples illustrate small, carefully designed steps toward the development of virtual books and writing implements. Combined with the more thought-provoking examples above, they suggest that we can begin to ask a new set of design questions regarding future reading and writing environments.
What should a "smart book" do? Learn your reading level and adjust itself accordingly? Shouldn't your word processor learn your writing style and bring into play a set of tools adjusted to your habits and needs? (In point of fact, four major software publishers have announced development plans along these lines.) How do you turn the page of a virtual book? With a wave of your hand? Does it even have pages? Can the notion of a book, set in type, with a fixed point of view, be expanded to include a more fluid entity, one that is shaped by each reader, with the construction of meaning derived from conscious and explicit interactions between the text, the reader, and previous readers who have become contributing authors? What about sound tracks for books?
Akram Midani, Dean of Fine Arts at Carnegie-Mellon University, notes that our involvement with new technologies generally moves from an ambivalent relationship with augmented abilities to the "dawning of irreversible change." Think of word processing in this regard. In the teaching of writing, there are still wide-spread uncertainties about its value, its effects on writing, the manner in which it should be taught, and the ancillary computer-based tools that should be used in conjunction with it (prewriting software, spelling and style checkers, keyboarding tutorials). Yet, are there many people who regularly use a word processor who would willingly give it up?
For all the excitement and positive potential that VR developments have engendered, there are appropriate cautionary notes. Four of the common issues often brought up in extended discussions of VR are these: The Reality Question, The Addiction Question, The Sex Question, and the Good and Evil Question. VR seems to make even more "real" some of people's concerns that digitizing our world deconstructs and decontextualizes it in ways that threaten and diminish our lives. Others worry that the immersive effects will be even more seductive than television, and they worry that the entertainment and advertising industries will drive and control the concepts and uses of the technology. In a related concern, there are those who are concerned about gender bias in the development and use of VR technology, in the consequences of VR pornography, and in the more general issue of defining sexual "relations" in a medium that for some people must by its nature involve a contradiction in terms. And in a broader context, there are those who wonder at the "wisdom" of allowing people to wander through a digital Ten Commandments, breaking them in any order and to any degree the individual chooses.
Virtual realities provide a very curious set of technologies, ones that have implications for the basic tools and substance of education. Barlow, quoted above, felt himself to be a "traveler in a realm what will ultimately be bounded only by human imagination" -- which for some people is the good news, while for others it's the bad news. At this point, the territory of virtual realities is a kind of unreal estate. Nevertheless, those who are exploring it are already helping expand and enrich our expectations and visions for integrating technology into our lives. These new technologies -- and the redefinitions of basic skills they will engender -- will provide new challenges for all of us who have a vested interest in adapting to technology and in adaptive technology.
Go to previous article
Go to next article
Return to the 1994 VR Table of Contents
Return to the Table of Proceedings