Feeling

Algorithm

Usability

Gesture recognition

Social psychology

Sketch recognition

Augmented reality

Computer graphics (images)

Computer science

Virtual reality

Gesture

Psychology

Optical head-mounted display

Mixed reality

Pointer (user interface)

Artificial intelligence

Sketch

Human–computer interaction

Sensory cue

Computer vision

Multimedia

Many researchers have studied various visual communication cues (e.g. pointer, sketching, and hand gesture) in Mixed Reality remote collaboration systems for real-world tasks. However, the effect of combining them has not been so well explored. We studied the effect of these cues in four combinations: hand only, hand + pointer, hand + sketch, and hand + pointer + sketch, with three problem tasks: Lego, Tangram, and Origami. The study results showed that the participants completed the task significantly faster and felt a significantly higher level of usability when the sketch cue is added to the hand gesture cue, but not with adding the pointer cue. Participants also preferred the combinations including hand and sketch cues over the other combinations. However, using additional cues (pointer or sketch) increased the perceived mental effort and did not improve the feeling of co-presence. We discuss the implications of these results and future research directions.

Evaluating the Combination of Visual Communication Cues for HMD-based Mixed Reality Remote Collaboration