Exploring New Modalities for CrowdRE: Indicators for Explanation Needs

Abstract

As software systems become more complex, end users' needs for explanation of certain system aspects also increase. A useful method for eliciting the needs of large user groups are common CrowRE methods, such as feedback mining in app reviews. However, these methods are not ideal for addressing the need for explanation, as minor issues rarely result in explicit feedback. To address this, we seek to explore new modalities for identifying the need for explanations among large user groups. To this end, we conducted an exploratory user study in which we observed 30 participants as they encountered situations in which they needed explanations within a software system. We identified behavioral patterns within the system, such as click spamming, that occur frequently in connection with the need for explanations. Physical reactions such as frowning or verbal expression were also observed in situations where an explanation was needed. Our vision is that in the future, after some validation steps and further refinement, these behavior patterns can be used in deployed systems as needs indicators. This would enable the automatic collection of explanation needs during runtime using telemetry data or other monitoring devices.