IEEE Consumer Electronics Magazine - March 2018 - 87

the three main directions. Both the tracking parameters
(input) and force-feedback parameters (output) can be
accessed and prescribed using the OpenHaptic libraries. For
this reason, it is possible to acquire the position and the attitude of the end effector and impose a reaction force to it.
This combination of position, attitude, and forces provides
many possibilities for implementing a high level of realism
and interactivity.
Although the actual stylus coordinates can be implemented into the AR scene, as confirmed by previous investigations
dealing with precise tracking in AR [23],[24], we prefer to
introduce a linear mapping M between the actual stylus coordinates (world coordinates p W h and that of the augmented
scene ^p ARh, to easily span the entire view frustum, independently from its dimensions. The mapping relationship can be
written in matrix form as
p AR = Mp W .

(1)

In this way, the stylus behaves like a spatial mouse.
According to this strategy, the real device is not an object of
the augmented scene; instead, the position and the attitude
of the stylus control the position and attitude of a virtual
replica of the stylus in real time.

FORCE-FEEDBACK INTERACTION
Another important addition in the augmented scene is its
capability to generate a force feedback. As noted, the Geomagic Touch X is also able to exert a three-component force
to its stylus. Since the stylus is grabbed by the user, the reaction force is transferred to the user's hand and interpreted as a
tactile sensation. The device is highly programable, allowing
for the possibility to create a variety of force-feedback
effects. For the specific application, we include two types of
feedback. The first applies to the interaction with virtual
objects in the scene. Since the virtual objects are mere graphical content, in standard AR implementation, the interaction
to the user is limited to visual effects. In the real world, if one
touches an object, he or she can feel the physical shape
thanks to the tactile feedback, which is generated by the contact force between the fingers and the object's surfaces. For
this reason, each object in the scene needs to produce a contact reaction as the user approaches the surface.
The interface between the real world and the augmented
scene is the haptic device stylus; therefore, the occurrence of
contact between the stylus and the virtual objects must be
checked. For this purpose, the distance between the stylus tip
and the surfaces of all of the active bodies in the scene are
continuously monitored. When the distance becomes negative
(i.e., a penetration d c is detected) a reaction force fc, n is generated for preventing the overlapping between geometries.
The direction of the reaction force is parallel to the common
normal vector n c at the contact location and the amplitude of
the force is set proportional to the distance of penetration
multiplied by a penalty factor k:
fc, n = k d c n c , d c $ n c 1 0.

(2)

All of the software necessary to
manage the interface input and
output are developed in C++
language by using the Microsoft
Visual Studio 2010 suite.
In the case of a rough surface, the contribution of the friction force fc, t can be also evaluated as
fc, t = - n fc, n v c,

(3)

where n is the dynamic friction coefficient and v c is the relative sliding velocity at the contact point. Combining (2) and
(3), the overall feedback contact force at the stylus can be
computed as
fc = fc, n + fc, t .

(4)

The second type of feedback relates to the helpers. They are
not physical objects; instead, they are graphical supporting
bodies or conceptual entities that guide the user through the
various modeling tasks. One of the simplest helpers is known
as the object snap, which can facilitate the pointing of points,
lines, edges, or other geometrical entities. When the user
approaches the entity to be selected, he or she is attracted by it,
so he or she can reach the exact location of the desired entity.
The snap is common in computer applications dealing with
geometries. Thanks to the force feedback, it is possible to generate the attraction force by using the haptic device. The correct definition of the snap requires a distance of influence d max
and an attraction gain k s and produces a feedback force fs computed as a piecewise function depending on the distance d s
between the stylus tip p tip and the point p to be snapped, where
fs = '

d s 2 d max
,
d s # d max

0
- ks ds

(5)

and
d s = p tip - p.

(6)

Virtual rulers can be also drawn in the augmented scene to
have a visual aid about the distance along the main direction. In
the same ways, the addition of virtual lines to the augmented
scene may help the translation along specific directions. In this
case, the force feedback fs, which attracts the stylus tip p tip
towards a spatial line , ^uh = vu + a can be computed in the
same way as (5), but the distance in this case is given by
ds =

v # ^p tip - a h
,
v

(7)

where v is the direction vector of the line and a is a point
on it.
It is easy to verify that, in this second case, the direction
of the force feedback is only orthogonal to the line to be
march 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - March 2018