Go to previous article
Go to next article
Return to the 1994 VR Table of Contents
By: David D. Andaleon
Sandia National Laboratories
P. O. Box 969, MS 9103
Livermore, California, USA 94551-0969
Phone: (510) 294-1552, FAX: (510) 294-1377
Creve C. Maples, A. Keith Miller & Sharon A.
Sandia National Laboratories
Albuquerque, NM 87185
Virtual reality software tools and training applications at Sandia National Laboratories provide a core capability to address the national need for assistive technologies.
Sandia National Laboratories has the responsibility to provide technical solutions for national problems. Throughout its forty-five year history Sandia has developed extensive capabilities in sensor technology, computer codes, modeling & simulation, etc. With more than 30 million Americans with disabilities, Sandia is investigating ways to focus these capabilities to address the national need for assistive technologies.
Advancements in assistive technology include the application of virtual reality(VR) to train & assist persons with disabilities. SandiaÆs VR technologies are applied in hypermedia training, skills training, field exercise planning, etc. Sandia has also developed an open-architecture VR environment that is both platform- and device-independent. This VR environment enables the user to adaptively change the human-computer interaction. Many of these VR activities have obvious dual-use applications in assistive technology.
The key activities for identifying areas of opportunity will be: (1) determining the specific needs of persons with disabilities, and (2) identifying existing research in VR-based assistive technologies. An effective way to accomplish these activities is through collaboration with VR and assistive technology institutions. Through technology transfer programs, such as the small business initiative, Sandia can collaborate with private industry and utilize its capabilities for advancement in assistive technology.
Key Words: Sandia National Laboratories, Department of Energy, Virtual Reality, Training, Assistive Technologies, Hypermedia
For more than forty years Sandia National Laboratories, in close cooperation with it's sister DOE laboratories, has been entrusted with the design and stewardship of the United States nuclear stockpile. The U.S. public, indeed, the world community, has demanded that nuclear devices be absolutely safe, while military necessity demand that they be absolutely reliable. Such responsibility requires that some of the best engineering talent in the United States be focused on research, development, design, production and assessment of high quality components and reliable, fail-safe systems.
Recent significant international events have afforded some welcomed relief from the previous intense focus on nuclear weapon design, production, application, and protection, and have spawned an opportunity for our laboratories to also consider other pressing national and global issues. The opportunity to thoughtfully seek dual or multiple uses for the many advancing technologies that assure this country a nuclear weapons capability holds hopeful promise for many sectors of our society. Providing cost-effective health care to the US population is an example of rising national priority. Another related example of rising awareness is the right of the disabled, approximately ten percent of the U.S. population, to have equal access and opportunity to participate in our society. To achieve adequate access often requires either augmented communication, or mobility assistance, or both. We see a particular opportunity to adapt the methods based in virtual reality environments used to train weapons assemblers, and test equipment operators to be readily adaptable to provide equipment that will help mobility impaired people to select the apparatus for their needs, to learn how to operate their selected equipment in a safe, virtual environment, or to demonstrate degree of accessibility of a facility, either existing or one in the planning stages.
The pursuit of our missions has required the interaction of many cross-cutting disciplines in unique ways. Sandia has long relied on massive simulation, modeling, and animation to thoroughly understand the probable outcomes of sophisticated experiments or the operational characteristics of complex systems. The training of personnel for both normal and emergency operations of experimental reactors is an obvious application for virtual reality. Initiatives to significantly reduce the nationÆs stockpile demand additional training exercises, methods & tools to accommodate the increased volume of weapons slated for dismantlement. Operators of tele-robotic vehicles designed to retrieve equipment and personnel from hostile environments must first learn how to operate those vehicles before actually embarking on a critical mission. Much or all of their training is often accomplished in a virtual environment.
Overview of VR research at Sandia
Sandia has been developing and applying VR technology for training, scientific visualization and design tools for roughly two years. Although new in this arena Sandia researchers in expert systems, real-time computing, computer-aided engineering and modeling & simulation have reapplied their skills to develop VR technologies. The success of a training tool is dependent on system design, implementation and assessment. Sandia's expertise in system engineering & integration is a key element of its VR research for training applications.
The research in VR can be partitioned into two areas: (1) applications, and (2) software & hardware development. The applications area focuses on identifying value-added applications of VR technology; the software & hardware area focuses on the basic toolset (computers, VR peripherals and software) for developing applications. This paper describes VR research in these areas by the following groups:
The Center for Exploratory Systems & Program Development is developing training systems that utilize VR technology and specialized hardware. This centerÆs primary focus is to utilize VR technology for weapon disassembly and equipment skills training.
The Intelligent Systems and Robotics Center is developing the virtual reality and intelligent simulation tools needed to apply VR to applications such as situational and task training and teleoperation of robots and sensors.
The Computational/Computer Sciences & Math Center is developing synthetic environment tools to assist in the interpretation of abstract and/or multi-dimensional data. These tools emphasize consistency between cognitive and subcognitive perception stimuli.
These three groups collaborate on VR projects. Some of the research areas are highlighted below:
Skills Training: The primary focus of VR peripheral development is to provide visual feedback with head-mounted displays, head coupled displays, stereo glasses, etc. The second and rapidly growing research area is in aural feedback, e.g., spatial sound. In fact many commercially-available VR peripherals provide relatively good quality visual and aural feedback. However, for skills training tactile and force feedback, which can significantly improve the effectiveness of a training tool, leaves much to be desired. For many training applications, the tactile feedback, force feedback and force reflection can be afforded by instrumented training versions of the system or equipment.
A system is being developed to train operators on pitch & yaw tracking telescope mounts that are instrumented to generate corresponding visual and aural cues. Tactile and force feedback is realized through the actual operation of the mount. Pitch and yaw information is sent to a computer system that generates visual and aural feedback to the operator.
In cases where the participant must interact with virtual objects directly with his or her hands any tactile or force feedback must be realized through a glove based system. Sandia is building on its smart structures materials experience, e.g., electrorheological gels, to advance the area of tactile feedback. Sandia is also building on its telerobotics experience to provide "virtual kinesthetics," i.e., force feedback and reflection.
Often times the normal operation of the equipment will interfere with providing a virtual training environment. In these cases the functionality of the equipment must be simulated by computer-controlled motion platforms, etc. Sandia has experience in modeling the missile dynamics inputs to an in situ guidance computer to assure that the control system generates appropriate response signals. These same techniques can be applied to training on the use of mobility aids, such as motorized wheelchairs. Model-based motion simulators, using multiple degree-of-freedom platforms, permits flexibility to train using models of various wheelchairs with minimal changes to the hardware configuration, e.g., various control devices. Computer generated visual and aural feedback supplements the motion platforms tactile and force feedback. The wheelchair training tool becomes an aid for improving the wheelchairs design when models of prototype or pre-production wheelchairs are available.
Situational Training: The system is being developed which uses VR to support non-proliferation treaty compliance. The system is used to train the escorts and inspectors of nuclear facilities subject to inspection under this treaty. The escort/trainee and inspector/trainer share a common virtual model of a DOE facility. Each participant can manipulate the objects in the VR. The trainer attempts to violate security by touching or viewing classified objects. The system automatically monitors the trainee's performance and provides a log at the end of a session. The underlying techniques developed for this project may be used to develop situational training systems for use in other areas, such as rehabilitation and evaluation of disabilities.
Hypermedia: Work is being done in the integration of VR with hypermedia. The goal is to create a complete training environment, for both situational and task training. VR is used to create a realistic simulation of the training task and domain. This simulation serves as the primary training tool. If, at any point, the trainee desires additional information, s/he may access text, video sequences, still photos, and other information related to the task or situation being trained. The VR model serves as the primary navigation tool through this additional data. Future extensions to this work will include the integration of intelligent agents for advising and tutoring, again accessed via the virtual environment.
Interaction with the system is via vocal commands and audio feedback, allowing access to computer-based information without the necessity of being able to use a keyboard and mouse.
Robotics & Automation
A system has been developed which allows programming and monitoring robot systems. VR techniques are used to allow the operator to generate programs by interacting, at the task level, with a graphical model of the actual robotics system. The operator may move around in the virtual environment, using voice commands and hand gestures to program accurate graphical models of actual robots. These task-level commands then automatically generate device-level programs to be down-loaded to the robots. The operator may also preview these operations for correctness via simulation and edit them interactively before sending them to the robotics devices. Such a system could be used to control robot devices used for handicapped assistance.
The goal of the MUSE project (Multi-dimensional, User-oriented, Synthetic Environment) is to develop an open software environment that permits users to describe input information (data, models, etc.) and then able to utilize a highly interactive, modular environment to help then visualize, explore, analyze, and understand information. Specific criteria for the system are:
Sandia's capabilities in sensors, modeling & simulation, expert systems and real-time computing provide an excellent technology base for virtual reality applications and tool development. With a mission to provide technical solutions to national problems, Sandia is well positioned to develop technologies, including virtual reality, to assist persons with disabilities and to prevent disabilities.
This work was supported by the Department of Energy under Contract #DE-AC04-94AL85000.
Go to previous article
Go to next article
Return to the 1994 VR Table of Contents
Return to the Table of Proceedings