1. MSc thesis
“Augmented freehand pointing on Wall Size Displays”
Abstract
Nowadays, there is a clear trend in the industry towards larger displays. Wall Size Displays (WSDs)
can be used in multiple occasions, from conference meeting rooms and large public displays to
private and public entertainment systems. For enhancing the user experience and performance
when interacting with large displays, the newest technological trends like Augmented Reality (AR)
can be embodied in the interaction. The present project is an attempt to answer whether a better
level in both the user experience and performance can be achieved by using Augmented Reality in
interaction with Wall Size Displays.
A WSD consisting of 12 HD projectors was used with a total of 7680 x 3240 pixels and resolution of
around 68pixels per inch. Distant freehand gestures interaction with marker-based tracking from the
OptiTrack motion capture system was employed, simulating the basic scenario of point & click. The
Optical-see-through head mounted display Epson Moverio BT-200 smart-glasses was used as AR
device, and the augmentation feedback was a laser beam. Three User Interfaces were implemented
and compared, the Simple, Combined and Augmented Interfaces. The Simple User-Interface acted as
the control interface, where the users were able to point & click only with the assistance of the
cursor. The Combined User-Interface differed from the Simple by incorporating AR, so the users
were able to interact with the assistance of both the cursor and the laser beam. The third interface
was the “Augmented User-Interface”, where the interaction was performed with an invisible cursor,
only through the AR glasses.For evaluating and comparing the performance of the interfaces, 12
participants followed a multidirectional pointing task as described in the ISO9241-9 standard. The
task for each interface was divided into 9 conditions with different Indices of Difficulty.
Several limitations were set on the experiment because of the glasses’ features and performance.
Nonetheless, a positive outcome for the augmentation’s effect was observed through the Combined
Interface’s performance and subjective assessment. The Mean Movement Times and mean error
rates in the different conditions tended to be overall better compared to the Simple, and especially
in conditions with big and medium targets. Regarding the Augmented Interface, its performance was
not considerably poorer than of the rest. Exempting the small targets which affected greatly the
interaction in this interface, its performance in conditions with big and medium targets was
comparable to the one of the other interfaces. The Augmented Interface behaved satisfactorily for
pointing and clicking in big and medium targets eliminating adequately the cursor, showing a
potential for solving multi-user interaction problems.