The impact of manipulative content on human performance in augmented reality

Abstract

Extended Reality is the spectrum of spaces and experiences, both virtual and augmented which include both Augmented Reality (AR) and Virtual Reality (VR). Of the two categories, Optical See-Through (OST) Augmented Reality is beginning to be used more widely in the public domain. However, addressing manipulative content is necessary for the widespread adoption of OST AR technology. Extended Reality (XR) Devices have had many vulnerabilities identified in previous works that may make them susceptible to the introduction of manipulative content, which an attacker may be able to use for a variety of purposes. For instance, in a cybersecurity context, attackers might try to influence and reduce user performance by changing the quality of AR information, introducing misleading content, irrelevant data, and other adverse factors. This may allow the attackers to control user behavior, slow down or stop important tasks performed in XR or to annoy or otherwise adversely affect the mental state of the XR user. This research investigates how helpful, misleading, and irrelevant information in OST AR affects human performance. The study used a memory task and employed a repeated measures design involving 19 participants. The findings revealed that the participants needed more time to complete the task when presented with irrelevant information compared to when they had access to useful AR information or when AR content was not presented. In addition, helpful AR information allowed users to complete the task more effectively with fewer errors than irrelevant and misleading AR information. The results suggest that AR enhances user memory, enabling them to perform tasks more efficiently. Moreover, when malicious information is introduced, manipulative content can effectively increase the decision-making time of their targets by disrupting memory-based judgments.