1993 VR Conference Proceedings

Go to previous article 
Go to next article 
Go to Table of Contents for 1993 Virtual Reality Conference 

Virtual Realities: From the Concrete to the Barely Imaginable

Stephen Marcus
Graduate School of Education
University of California, Santa Barbara

Suddenly I don't have a body anymore. At least I know where I left it. It's in a room in a building in a town in California. But I--or "I"--am in cyberspace, a universe churned up from computer code, then fed into my eyes by a set of goggles through whose twin videoscreens I see this new world. All that remains of my corporeal self is a glowing, golden hand floating before me like Macbeth's dagger. I point my finger and drift down its length to the bookshelf on the office wall. John Perry Barlow.

More and more these days I find myself giving talks that leave me speechless. I hardly know what to say about the extraordinary developments in high-tech tools that often far outdistance our sense of how best to understand them or best adapt them to teaching and learning.

Forward to the Basics

At various laboratories and art installations around the country, and in some malls and video arcades, you can now enter worlds that resemble the ones you're generally familiar with, but in which the laws of the universe are sometimes modified or eliminated. To do this, you typically wear special headgear that tracks head movements and provides a three-dimensional image (a kind of Walkman for the eyes). There is also a glove or other device that tracks the position and configuration of your hand. Both pieces of apparatus are connected to a computer that provides the visual displays and responds to your head and hand movements (and sometimes provides sounds, pressure, and resistance to movement).

These technologies are used to create a "virtual reality," a simulation that you seem to enter physically. It's a kind of "deep medium." Randall Walser, of the Autodesk Research Lab, suggests that if text "tells, and video and film show, then a virtual reality embodies the world it creates." Virtual realities dissolve the line between the interface and the innerface of the worlds you wish to explore.

John Perry Barlow, who's experience with the AutoDesk system introduces this discussion, tries to "grab a book, but my hand passes through it." After some guidance, he succeeds: "When I move my hand again the book remains embedded in it. I open my hand and withdraw it. The book remains suspended above the shelf." Since the only part of himself he sees in this new world is "a glowing, golden hand," he realizes that he doesn't "seem to have a location exactly. In this pulsating new landscape, I've been reduced to a point of view. The whole subject of 'me' yawns into a chasm of interesting questions."

In a very different setup, David Rokeby's "A Very Nervous System," your body's movements (with no glove or goggles) are tracked by video cameras and translated into music. Space itself is now the interface between you and the computer; space has become a medium in itself. People who have experienced this sort of environment sometimes note that the relationship between cause and effect seems to disappear. Rob Appleford, for example, noted that, "[As I walked back to the university] a truck splashed through a puddle, and I caught myself trying to make that neat noise again by retracing my steps."

Rokeby's system has been used to allow quadriplegics to perform with professional jazz musicians. In one case, the waving of fingertips plays a virtual piano. In another case, a mouth stick moving slightly in the air becomes a percussion instrument, playing virtual drums.

A related kind of system, "Mandala," from Vivid Effects, also has applications for people with special needs. In this case, you are in front of a video camera with a blank background behind you. Your image is transferred into a computer screen that has special images on it (e.g., musical instruments, floating alphabet letters, or a hockey rink). As you move in front of the camera, your image on the screen can interact with the objects there: playing the instruments, grabbing letters to form words, blocking shots. You (or is it your image?) have become a virtual icon, interacting with other icons on the screen.

A different goal has been addressed by the MCC consortium in Texas, where they've developed ways to represent large and complex bodies of knowledge in a computer-generated virtual space. In this case, navigating through a body of knowledge involves being able to yaw, pitch, and roll as if you were flying a helicopter, a rather different set of skills than knowing how to turn pages or to use an index or table of contents.

Closer to Home

It's hard to communicate, in print, the look and feel of entering virtual worlds. (Can you easily describe to a non-reader what it feels like to be "lost in a good book"?) Having tried, however, to make strange things familiar, it might be instructive at this point to make some familiar things strange.

Consider this. Text simulates thought. It's an artifact, representing with its own particular richness and its own unique limitations the ineffably complex workings of the human mind going about its business of making sense of things. Text is a version of the thought that the text represents. It's a working model of what's on our minds. As with other kinds of simulations, we can interact with and affect text (aka reading and writing). Text is virtual thought.

We've long since become accustomed to a range of materials that embody text, paper being the predominant one (which some people, interestingly, equate with being "carved in stone").

Now consider this. A word processor creates virtual paper. The technology, in fact, fills a new medium with an old content. We tend to think of a word processor as a kind of way-station on

the way to a printout, "hard copy." We imagine there's a sort of paper scroll moving past the "window" of the monitor screen. This simulated paper has, however, powers and abilities far beyond those of normal paper. The "videotext" sometimes blinks, ripples, and slides. It can disappear and reappear. It can change its shape and sometimes its color. It can have embedded in it "buttons" (and I am talking about word processors here) that will play sounds and recorded voices or show moving pictures. This sort of virtual paper is less and less designed to be "printed out" (although more and more we're hearing the phrase, "print to video"). Increasingly, this sort of document's home is in the computer environment in which it was created.

Many people's thinking includes pictures and sounds as well as words--unless, like Einstein, you also think in colored patterns and muscle sensations (contrary to my high school English teacher's dictum that "if you can't put it into words, you can't be thinking it"). Thus, these kinds of technology-enabled multimedia and "hypermedia" environments, ones that allow a variety of "documents" to be stored, linked, intermixed, displayed, and altered by the "writer" and "reader," are steps toward the creation of virtual texts.

And Down the Road

There are, to be sure, examples of "virtual books" that are currently available commercially. The illustrated children's stories in the Discis collection, on CD ROM discs, will read themselves to you in English or Spanish, in word or phrase groupings of your choice. They will provide you with definitions of each word and will remember the words you didn't know. You can have the books instantly "rewritten" in different typographies. The recently released Expanded Books, from the Voyager company, provide moving images and sounds to accompany the text, along with special text-searching and annotating features. The "books" are stored on computer disk, for use with Apple Computer's PowerBook series of computers.

These two examples illustrate small, carefully designed steps toward the development of virtual books and writing implements. Combined with the more thought-provoking examples above, they suggest that we can begin to ask a new set of design questions regarding future reading and writing environments.

What should a "smart book" do? Learn your reading level and adjust itself accordingly? Shouldn't your word processor learn your writing style and bring into play a set of tools adjusted to your habits and needs? (In point of fact, four major software publishers have announced development plans along these lines.) How do you turn the page of a virtual book? With a wave of your hand? Does it even have pages? Can the notion of a book, set in type, with a fixed point of view, be expanded to include a more fluid entity, one that is shaped by each reader, with the construction of meaning derived from conscious and explicit interactions between the text, the reader, and previous readers who have become contributing authors? What about sound tracks for books?

Akram Midani, Dean of Fine Arts at Carnegie-Mellon University, notes that our involvement with new technologies generally moves from an ambivalent relationship with augmented abilities to the "dawning of irreversible change." Think of word processing in this regard. In the teaching of writing, there are still wide-spread uncertainties about its value, its effects on writing, the manner in which it should be taught, and the ancillary computer-based tools that should be used in conjunction with it (prewriting software, spelling and style checkers, keyboarding tutorials). Yet, are there many people who regularly use a word processor who would willingly give it up?

Virtual realities provide a very curious set of technologies, ones that have implications for the basic tools and substance of education. Barlow, quoted above, felt himself to be a "traveler in a realm what will ultimately be bounded only by human imagination." At this point, the territory of virtual realities is a kind of unreal estate. Nevertheless, those who are exploring it are already helping expand and enrich our expectations and visions for integrating technology into reading and writing. These new technologies--and the redefinitions of basic skills they will engender--will provide new challenges for all of us who have a vested interest in adapting to technology and in adaptive technology. We will gain fresh perspectives on the insight provided by Allan Brightman, at Apple Computer, who noted that technology helps "ordinary people do extraordinary things, and extraordinary people do ordinary things."


Go to previous article 
Go to next article 
Go to Table of Contents for 1993 Virtual Reality Conference 
Return to the Table of Proceedings 

Reprinted with author(s) permission. Author(s) retain copyright.