Coral: Contextual Gists for Blind and Low Vision Screen Reader Users' Understanding of Dynamic User Interfaces

Abstract

Blind and low vision (BLV) screen reader users construct mental models of user interfaces (UIs) through incremental screen reader interaction, a time-consuming and cognitively demanding process complicated by modern interfaces that may not convey dynamic content accessibly. We interviewed eleven BLV screen reader users about UI understanding and derived design objectives that informed Coral, a context-aware browser extension. Coral synthesizes interface and user context to generate screen reader-narrated gists which holistically notify users of interface states and changes relevant to their likely goals and ongoing interaction. In a comparative evaluation with eight BLV screen reader users, participants used Coral to form initial expectations and interpret interface changes, and reported spending less time and effort manually verifying interaction outcomes. Together, our findings provide deeper insight into how BLV screen reader users navigate gaps in their UI understanding, and how intelligent support that combines interface and user context can bridge these gaps.