Augmented reality

Pure mathematics

Mixed reality

Mathematics

Virtual image

Human–computer interaction

Computer graphics (images)

Computer-mediated reality

Virtual reality

Immersion (mathematics)

Computer science

Computer vision

Journal of Robotics Networking and Artificial Life

Augmented Reality (AR) is a technique of superimposing information generated by a computer on perceptual information that we receive from real space.The combination of AR technology with many contents creates new type of applications and AR is used in many fields, including education, medicine and entertainment [1-3].More recently, techniques for realizing interaction between a user and virtual objects in AR have garnered much attention.Therefore, it is expected that these technologies will be used in various applications in the future.On the other hand, most of the existing studies focused on interactions using rigid virtual objects.However, in this investigation, we aim to realize a flexible virtual object that can be deformedmuch like clay-and use it in a wider range of applications.This paper introduces examples of flexible virtual objects.Kato et al. [4] introduced flexible body virtual objects by using Bullet Physics (or physics engine), reflecting the action of crushing on the virtual object with the user's bare hand in the virtual space, and realized the deformation manipulation of virtual objects by interaction with the user.Suzuki et al. [5] depicted the moving manipulation of rigid body virtual objects through an interaction between the user and virtual objects.However, after deforming the virtual object, moving or rejoining the virtual object while maintaining the original shape was not possible.To the best of our knowledge, a series of movements that deformed, moved, or joined virtual objects were not realized.This system consists of a Web camera for real image acquisition and marker recognition, the Leap Motion Controller for obtaining 3D coordinates of fingers, and PC for performing arithmetic processing and video output.In this system, we use network programming to prevent interference between the Leap Motion Controller and the Web camera.This system acquires the 3D coordinates of fingers with the Leap Motion Controller on the server side.In addition, this system acquires images with the Web camera, and performs physical calculation, and video output on the client side.

Intuitive Virtual Objects Manipulation in Augmented Reality: Interaction between User’s Hand and Virtual Objects