37th Annual CSUN Assistive Technology Conference Has Concluded

Augmented Reality Assistant for Independent Living

Date & Time

Friday, March 18, 2022 - 9:20 AM PST

Location

Elite 1-3

Description

Aging adults with cognitive impairments suffer from detrimental cognitive functions such as working memory and planning tasks. Specifically, adults experiencing early stages of cognitive declines and impairments (e.g., Alzheimer's disease and related dementias, mild cognitive impairments) have difficulty with executive functioning (e.g., executive set-shifting, speed of processing) that inhibit them from performing instrumental activities of daily living (IADLs). We build a prototype—Augmented Reality Machine Operatives Research (ARMOR) to promote mental functioning and detection of dementias. ARMOR uses augmented reality (AR) and machine learning (ML) to create a centralized program to understand a user's living arrangement and assist them throughout their day. Specifically, we are integrating the spatial mapping of AR and dynamic learning of ML to create features to foster greater independence for aging adults and their caregivers. The program would operate as another digital caretaker and resemble a structure found in modern-day video game AI. The program would adapt to a user's life by integrating itself into most people's current technology ecosystem. It can connect to a user's medical data to generate a schedule on when medication needs to be taken or picked up. As for other daily necessities, it will also read messages from family members and provide reminders for any events found within the conversation. It will also use facial recognition to identify individuals and pull up any content relating to them. And by taking the information it knows about its user, the program will be able to retrieve health recommendations from professional and family caregivers to create an optimized schedule on when and what to eat. When the program is unsure of anything, it will contact a relative or caretaker to edit and save the plan manually. All this information will be neatly presented in AR as it provides lots of flexibility in displaying information. By equipping the headset, the participant will have a list of tasks to complete for the day in an ordered manner and be given clear directions on how to facilitate the completion of each lesson. That is, arrows or footsteps will help guide aging adults to the correct area to complete their next task and glowing indicators showing the required items to meet the function in the assigned area. Likewise, various reminders can be activated, such as informative graphics presented related to the next activity on the list and a clock showing the task's completion time. An example of this would be glowing arrows guiding the subject to the kitchen, a bright border around the refrigerator, and a glowing frame on a prepared meal to accomplish the task of eating breakfast. We will implement a reward-type system that mimics markers found in games to bridge an inherent gap that may appear when communicating between a machine and a person. A reward of points will be assigned when successful completion of a task depends on the task's speed, successful completion, and accuracy. The goal is to provide relief to caretakers who assist these older adults in various IADLs to ensure they are still functioning despite impaired or reduced memory capacity. Regular quizzes can be administered as a challenge to the participant testing their memory to gauge their mental condition and provide an opportunity for augmented memory and planning functioning. The ultimate goal is to alleviate the caregiver burden of assisting in IALDs requirements and restore autonomy to the elderly patient.

Audience

Disability Specific
Healthcare & Rehabilitation
Research & Development

Audience Level

Intermediate

Session Summary (Abstract)

We will discuss the practical impact of augmented reality in assisting caregivers and dependents with daily tasks required for independent living. The primary application is in the healthcare and rehabilitation industry, providing tools to support the staff and the patients.

Session Type

General Track

Topics