AN
IMPROVED TOOL FOR PRACTICING FINGERSPELLING RECOGNITION
Presenter(s)
Rosalee
Wolfe
DePaul University
243 S. Wabash Ave
Chicago, IL 60631
Day Phone: 312 362 6248
Email: wolfe@cs.depaul.edu
Rosalee
Wolfe, Nora Alba, Stacey Billups, Mary Jo Davidson,
Cindy Dwyer, Diana Gorman Jamrozik*, Lori
Smallwood, Karen Alkoby, Lesley Carhart,
Damien Hinkle, Amy Hitt, Brian Kirchman,
Glenn Lancaster, John McDonald, Lindsay Semler,
Jerry Schnepp, Brent Shiver, Anna Suh, Jeffrey Young
School of Computer Science, DePaul University
*ASL-English Interpretation Department, Columbia
College, Chicago
asl@cs.depaul.edu
Our new fingerspelling
practice software displays realistic animations of fingerspelling,
including naturalistic transitions between letters that were previously
impossible.
Introduction
Fingerspelling
is an important component of American Sign Language (ASL) and is a necessary
skill for complete communication in sign [BATTISON 78]. Padden found that fingerspelling
makes up 7 to 10 percent of the signs produced in everyday ASL conversations [Mulrooney 02]. Fingerspelling
is useful for spelling proper nouns, technical terms, acronyms, initialized
signs and words from foreign languages. When fingerspelling,
people use their dominant hand to create a series of manual symbols, one
corresponding to each letter of the word. A person fluent in ASL
can produce fingerspelled words at a rate of four
characters per second.
In contrast,
the fingerspelling recognition rate for people
learning sign language is far lower. Acquisition of fingerspelling
recognition skills typically lags far behind other sign language skills
[GRUSHKIN 98] [SCHLEPER 03]. Wilcox surveyed sign language students, who
reported that recognizing fingerspelled words was the
toughest part of learning ASL [WILCOX 92].
“Fingerspelling is the first skill learned and
the last skill mastered” [
Achieving fingerspelling fluency requires the visual comprehension of
the manual representation of letters. One reason students experience difficulty
in fingerspelling recognition is its high rate of
symbol presentation. Most signs in ASL use no more than two hand symbols
[BATTISON 78], but fingerspelling uses as many
symbols as there are letters in a word.
An
additional barrier to improving fingerspelling
comprehension is the lack of resources for self-study. Several interactive
programs do exist that are capable of showing a series of static images of the
manual alphabet [VISION 93] [ASL 05] [GAY 01] [INSTITUTE 03], but none show the
motion between letters. These motions are essential in fingerspelling recognition. When interviewed, fluent
signers mention that they look not for individual letters, but for the “shape”
of a word [GROODE 92]. Watching a series of static images in sequence
will not help.
To solve the
problem of realistically displaying fingerspelling,
we have developed an approach that displays the transitions between letters. We
have developed new software which implements this technique as part of an
interactive learning tool for practicing fingerspelling
recognition. As Figure 1 demonstrates, this new technology displays the
transitions that naturally arise when a person produces each letter in
succession.
<img
ALT="static images (old approach) vs. realistic transitions (our
approach)"
src="http://asl.cs.depaul.edu/csun05/figure1.jpg">
Figure 1: T-U-N-A: comparing the previous and
improved approaches to fingerspelling display
Design Criteria
Three major user-centered design criteria guided
the software development:
1. Simple navigation and visual appeal
The most
challenging aspect for the user should be the recognition of fingerspelling, not negotiating the interface. To
cater to a variety of users, many of whom are not computer-proficient, the
interface layout should be consistent. The controls for basic options should be
visible at all times, so the user will know where they have been and where they
can go. An interface that is visually appealing and easy to use
encourages users to spend more time with the software.
2. Multiple levels of challenge
The software should accommodate all skill levels and give
users the opportunity to experience a variety of recognition challenges. Skills
improve with practice, so users should be able to start at their current
comfort level and progress to more challenging options. Each level should
include an adjustable speed control. By adjusting the content and presentation
in accordance with individual skills, the learning process is facilitated and
this can in turn improve user confidence by offering realistic goals [PALMER
03].
3. Review and self-quiz
Users need
opportunities to test their recognition skills. The design should offer
multiple-choice and fill-in-the-blank practice sessions and quizzes, each
unique to the level of difficulty. The software should provide a summary
of the number of correctly and incorrectly identified words. Giving the user
relevant information during practice sessions and quizzes will allow them to
analyze their own progress and identify areas of improvement [IKEDA 99].
The Fingerspelling
Tutor
We have
implemented the interface shown in Figure 2, which displays naturalistic
animations. The program runs on Intel-based PCs with Microsoft XP.
<img ALT="fingerspelling model and user controls"
src="http://asl.cs.depaul.edu/csun05/figure2.jpg">
Figure 2: Interface of Fingerspelling
Tutor
The software
offers multiple levels of challenge:
1. Alphabet
This level
displays each symbol of the manual alphabet in a simple chart, with both front
and back views. In the practice and quiz sessions, users guess the letter
and receive feedback on their response. At all levels, there is an unlimited number of repetitions in practice sessions,
but only one repetition in quiz sessions.
2. My First Fingerspeller
This level
introduces fingerspelling and presents words that are
only two or three letters long. The practice and quiz sessions are
multiple-choice. After the software fingerspells the word and the hand
returns to a neutral position, a set of four possibilities appear on the
screen. Users can choose an answer or click the “Repeat” button.
After receiving feedback, users can click “Show it again”, which is
helpful when the word was incorrectly identified. These buttons are also
available at the Intermediate and Advanced levels.
3. Intermediate
At the
intermediate level, users choose from a category (i.e. food, animals…), view a fingerspelled word, and select from four possible answers.
To provide a realistic challenge, the possible answers presented are of similar
length and contain the same first and/or last letter. The category gives a
context to the fingerspelled word which assists the
user in selecting the correct response.
4. Advanced
In the advanced level, practice and quiz sessions use a
“fill-in-the-blank” style of interaction.
At all
levels, users can access the “How Am I Doing?” button. This displays a
summary of a user’s responses, including feedback on
the number of words correctly identified as well as a list of
incorrect responses and the corresponding correct answer.
Testing the Software
After developing the software, we tested its
usability with 19 users, who were ASL students, tutors or teachers in an
accredited baccalaureate ASL-English Interpretation program.
In the test sessions, users were
asked to pick the option that corresponded with their current skill level
(Alphabet, My First Fingerspeller, Intermediate,
Advanced), and to explore the software. All 19 users began practicing fingerspelling recognition within one minute with no
assistance from the test facilitator. This is a strong indication that the
interface is intuitive and easy to use.
On the
post-test questionnaire, 17 of 19 users responded “Agree” or “Strongly Agree”
to the statement, “I would like to user the Fingerspelling
Tutor frequently.” To the statement, “The Fingerspelling
Tutor can help me increase my fingerspelling skills,”
17 of 19 users responded, “Agree” or “Strongly Agree.” These are good
indicators that people not yet fully fluent in signing will find the tool
useful.
Future Work
We will test the user opinion that the software
would help improve fingerspelling skills. We will do
this by evaluating the software in a classroom setting.
References
[ASL 05] ASL
Fingerspelling Quiz Site: http://asl.ms Accessed September 18, 2005.
[BATTISON
78] Robbin
Battison. Lexical Borrowing
in American Sign Language.
[GAY 01]
Greg Gay. GG Wiz’s FingerSpeller Site: http://www.iwaynet.net/~ggwiz/asl/ Accessed
September 20, 2005.
[GROODE 92]
Joyce Groode. Fingerspelling:
Expressive and Receptive Fluency. Videotape.
[INSTITUTE
03] Marvin Teaches Fingerspelling. Institute for Disabilities Research and Training, Inc. [software
CD].
[IKEDA 99] Ikeda, N. Language learning strategies with sound-hints in
computer-based drill. Journal of computer Assisted Learning,
15:4, 312-322.
[
[MULROONEY
02] Kristin J. Mulrooney, “Variation in ASL Fingerspelling”. In (Ceil Lucas, ed.)
Turn-Taking, Fingerspelling, and Contact in Signed
Languages, 3-23.
[PALMER 03]
Palmer, Ian. Developing On-line Instruction Courses
with E-learning Tools. Faulkner Information
Services, 2003.
[PATRIE 92]
Carol Patrie, “Fingerspelled
Word Recognition Skills in Sign Language Interpreters: A comparison Between
Novice and Experienced Interpreters.” Journal of
Interpretation. 5:1 (1992) 51-90.
[REGISTRY
04] Registry of Interpreters for the Deaf lists 31
[WILCOX 92]
[VISION 93] Visionware.
Fingerspelling & Numbers. CD.
Go to previous article
Go to next article
Return to 2006 Table of Contents