CollAboRation - Opportunities of augmented reality for novel interaction paradigms and communication research

This thesis investigates how augmented reality (AR) can be applied to a) improve interpersonal and human-computer collaboration, b) scientifically investigate collaboration in novel ways, and c) support accessible and reproducible research. First, perceptual processes that shape the foundation of AR-enabled communication research are introduced, followed by a variety of AR prototypes, their concepts, and technological requirements. The central part of this thesis is dedicated to the second funding phase of the interdisciplinary project Alignment in AR-based cooperation (AlARCo). During 57 trials split across 5 studies, 115 participants engaged in negotiations during a fictional dyadic city planning task by using (AR) mediation objects. Roughly 136 hours of video material and comparable amounts of sensor data had to be processed to create the AR-based Corpus (ARbC) which contains more than 42 hours of synchronized video, audio, and sensor data. Study conditions were chosen to investigate the baseline limitations induced by the used multimodal Augmented Reality-based Interception Interface (ARbInI) as well as effects of assistance and disturbance approaches. The results show that participants deal in different ways with disturbances but also suggest modalities that are not significantly influenced by the setting. For instance, on a verbal level, we have observed reduced overlap durations during turn transitions when users had been confronted with perceptual inconsistencies induced by ARbInI. Statistical differences concerning the use of head gestures, mediation object handling, body posture, and speech activity between AR and face-to-face conditions or between assisted and base conditions are limited to a few phenomena and do not seem to significantly hinder the negotiation process. This may be related to the speakers' assumption of normality, the recipients' wait & see strategy and the observed tendency of users' behavior and procedures to be influenced by whether a task can be solved collaboratively rather than if the used task solving procedures are objectively efficient. The interdisciplinary approach of AlARCo presented in this thesis also shows how ARbC can be prestructured with the help of data mining to ease qualitative analysis and identify interesting phase transition events. Additionally, results of conversation analysis could be used to create automated action classifiers based on interaction patterns and train deep neural networks to detect deictic gestures in the corpus video data based on analysts' annotations. Presented processing and analysis steps have been documented in Jupyter notebooks and can be reproduced interactively and altered on demand for transparency and to ease accessibility of the gathered research data. Furthermore, AR-related attention management concepts derived from insights gathered in AlARCo have been successfully applied to other contexts such as user assistance in everyday scenarios. For instance, an assistance system tailored for attention guidance in the cooking context has been created. However, the derived insights, modeled classifiers, trained neural networks and created prototypes represent a tiny step in the endeavor to create collaborative systems. Their creation and application processes show relevant considerations about what information should be presented in AR and how it should be integrated.