2006 Conference General Sessions

COMPASS: SOFTWARE FOR COMPUTER SKILLS ASSESSMENT

 

 

Presenter(s)
Heidi Koester
Koester Performance Research

2408 Antietam
Ann Arbor MI 48105
Day Phone: 734—663—4295
Fax: 734—663—8824
Email: hhk@umich.edu

INTRODUCTION
An accurate and efficient assessment of client abilities is a prerequisite for a
successful computer access intervention. To support effective access assessments, we have developed Compass, an easy—to—use software tool for measuring a client’s skills needed for
various kinds of computer interaction, such as text entry, mouse or pointer use, and information processing. Compass provides quantitative user performance data that
complements the subjective observations made by clients and practitioners. It automatically records speed and accuracy measures during task performance, leaving the clinician free to observe more subjective aspects of client abilities, needs, and preferences. Data are presented in an easy—to—understand format that can be used in reports and letters of justification.

PROCRAM DESCRIPTION
The current version of Compass includes eight skill tests in three input device domains (printing device use, text entry, and switch use). Each test is configurable to match it properly to the client’s needs. For example, the size and color of text and objects can be adjusted if necessary. These configurations are saved, allowing for efficient re—use in a subsequent session if desired. During a test run, data regarding the speed and accuracy of user actions are recorded. Following the test, Compass can generate a report, summarizing results for the test and providing trial-by—trial detail if desired. The currently available Compass tests are described in the list below.
1. Aim - Move mouse pointer into a target. Click (or dwell) to select target.
2. Drag — Move mouse pointer into a target. Click (or dwell) to select target. Drag
target to a destination. Release target.
3. Menu - Select a specific item from a menu bar.
4. Letter — Enter a specific letter.
5. Word - Enter a specific word.
6. Sentence - Enter a specific sentence.
7. Switch - Press and release a switch.
S. Scan — Select a letter using a row-column scan matrix.

USABIL1TY TESTING
Development of Compass has been guided by extensive user—centered research activities. System requirements were defined and refined through a survey of 103 clinicians and in- depth discussions with 25 more. In four formal usability trials, a total of 36 clinicians were able to perform a series of representative Compass activities with no external help and without major errors. In a recent beta test, the overall usability rating averaged 4.6 on a scale of 1 — 5 (5 being the best score), with a 95% confidence interval of [4.4, 4.7]. The overall usability rating was based on responses to 10 survey questions regarding ease of learning, efficiency of use, etc. Results from these tests support the conclusion that Compass has a high degree of utility and usability.

CLINICAL APPLICATION
Appropriate use of Compass helps a clinician to:
1. Diagnose difficulties with an existing interface; 2. Evaluate and compare performance with candidate access systems; 3. Plan training interventions; 4. Track changes in a client’s abilities over time; and 5. Measure the effectiveness of an intervention. Using Compass can enhance the objectivity and accuracy of access assessments as well as the efficiency and quality of associated reports.

Members of the project team who provide computer access services have successfully integrated Compass into their evaluations. Prior to meeting with a client, an entire session can be “scripted” by configuring multiple skill tests. During the session, skill tests can be easily duplicated or modified in response to observations of the client’s performance and behavior. An unanticipated benefit of Compass is that it allows the clinician to spend more time looking at the client and less time looking at the computer screen, because Compass provides detailed information about the client’s performance. This allows the clinician to focus on issues such as how much effort the client is exerting, the strategy the client is using, and the position the client is maintaining. After each test, the clinician can review the results with the client to clarify tradeoffs between different devices, configurations, and strategies. These results also become an integral part of the final report provided to third-party payers.

NEXT STEPS
Compass is currently being prepared for commercial release. We expect that it will be commercially available by the time of this presentation in early 2006.

ACKNOWLEDGMENTS
This work is supported by NIH grant 2R42 NS3625202A1. We also thank the clinicians who have responded to our surveys and participated in usability testing.
 

REFERENCES
1. Ashlock G., Koester H., LoPresti E., McMillan N., and Simpson R., “User-centered
Design of Software for Assessing Computer Usage Skills,” 26th Annual Conference on
Rehabilitation Engineering (RESNA), Atlanta, GA, June 2003.
2. Koester H., LoPresti, E. . “Compass: Software for Assessing Computer Usage Skills”, SUN Conference on Technology for People with Disabilities, Los Angeles, CA, March, 2003.
3. LoPresti E., Koester H., and McMillan N. “Tools for Assessing Computer Access Skills,” Proceedings of ASSETS 2002, New York: ACM.


Go to previous article
Go to next article
Return to 2006 Table of Contents


Reprinted with author(s) permission. Author(s) retain copyright