38th Annual CSUN Assistive Technology Conference Has Concluded
Task Aware Browsing: AI Assisted Web Access for Blind Users
- Date & Time
- Thursday, March 16, 2023 - 9:20 AM PST
- Platinum 2
A completely new approach for blind users to access web pages is presented. It uses machine learning and artificial intelligence to overcome the problems encountered by these users with conventional browsers and screen readers, particularly on an unfamiliar web site. Finding the location of key features, when accessing Internet web sites, is difficult and time consuming. This approach provides a method for finding active controls, and tracking and managing the results from invoking these controls. The new approach is called TAB: Task Aware Browsing, For a given vertical domain, e.g. shopping, travel, banking, etc., a set of keyboard commands is created that allow the user to perform the essential functions in that domain. The browser then uses a model generated from machine learning to convert each command into the appropriate actions for a learned web site. It filters the results of page updates to present the screen reader with only the essential portion of rendered page that is responsive to the command, eliminating excess clutter. Work performed to date indicates that machine learning techniques will allow this approach to work on a significant number of web sites. User testing demonstrates that it provides an appreciable improvement in the speed with which blind users can accomplish desired tasks.
This Presentation Link is provided by the Presenter(s) and not hosted by the Center on Disabilities at CSUN. The Center on Disabilities has confirmed, as of March 2, 2023, content linked is relevant to the presentation, but has not been reviewed for accessibility nor will the Center on Disabilities attempt to remediate any accessibility issues in the linked content. Please contact the Presenter(s) with any accessibility concerns.
- Higher Education
- Information & Communications Technology
- Research & Development
- Audience Level
- Session Summary (Abstract)
- AI can provide blind users with a command line interface to perform tasks on many web sites. This solves the common problem these users experience in navigating the web. Experimental results demonstrate the feasibility of this approach.
- Primary Topic
- Blind/Low Vision
- Secondary Topics
- Artificial Intelligence (AI) & Machine Learning (ML)
- Session Type
- Journal Track
- David Cane
Perkins School for the Blind