IEEE Systems, Man and Cybernetics Magazine - July 2018 - 10

Moderator: "Please,
perform the external
action."
(a)

(b)

Moderator: "Please,
perform the external
action."
(c)

Figure 3. the video source switching function in touchless mode: (a) the virtual environment, (b) a

demonstration of the changing action, and (c) the participant performs a task in the real-world environment.

a 4.3-cm diameter of each wheel and also considered the
elastic component of ropes. The angles can change according to the physical implementation.

most reliable tools involving hand tracking, having a high
level of accuracy [9]. The sensor is considered so accurate
that it is increasingly being used in hand rehabilitation [10].
Using a very simple proposed gesture allows the sensor to
consistently catch the motion, even during substantial illumination changes. Finally, the device has other interesting
features that make it suitable for the proposed project,
including lower cost, the ability to
be combined with the most popular
VR HMDs, and its small size.

Virtual Environment Versus Real World
When the surgeon is working inside the patient's body, his
or her focus is on the intrabody parts of the patient, but
sometimes he or she may need to
perform some external actions,
such as changing an instrument or
dabbi ng the i n ser t ion wound.
Currently, the new
Tests, Results with
While using an HMD, he or she
challenge is to
Surgeons, and Future Work
should remove the visor for a natuTo prove the effectiveness of the
ral visual interaction with the real
integrate these
IVRE system in surgical operaworld. Since this operation is untechnologies to
tions, we designed three different
comfortable, the IVRE system intesupport surgeons
tasks that are similar to real ac grates the Leap Motion in front of
tions performed during surgery.
the HMD, thus allowing the surin exploring and
For each, we applied the virtual
geon to switch, in touchless mode
managing a target
prototype into two-dimensional
(i.e., moving the hand from up to
(2-D) and 3-D environments, and
down and vice versa), the video
organ or tissue as if
we evaluated the results. Moresource stream from the endoscope
they were inside
over, we compared the proposed
cameras to the Leap Motion's one
the patient.
system with the most popular
to see the surrounding real world.
endoscopy techniques for correlatThe sensitivity and the distance
ing their ease of use. Twenty-one
threshold of the Leap Motion headendoscopic surgery experts partictracking recognition are calibrated
ipated in our experimental protocol.
for the camera switch function (Figure 3). In particular,
In the first test, we asked the surgeons to execute three
the user performs operations inside the virtual environcuts with surgical scissors to a metallic cable located in
ment [Figure 3(a)]; a moderator asks the user to execute an
front of the primary vision system (i.e., endoscope). The
external operation, so, he or she switches the video source
surgeons used the keyboard to move the scissors in the
by passing his or her hand in front of the HMD [Figure 3(b)].
virtual environment. In the 3-D case, we asked them to put
Finally, he or she can see the real world and he can execute
on the HMD, while in the 2-D case, they simply used a 2-D
the requested operation [Figure 3(c)].
monitor. In the first case, the head movements controlled
The switching action was chosen among numerous
the camera; in the second case, the camera was controlled
ideas and solutions (e.g., button or microphone). A survey
by an optical input device (mouse).
was distributed to gather the opinions of the engineers and
In the second test, we asked the surgeons to play a
surgeons involved in developing the system. The proposed
"minigame" similar to an endoscopic surgery environtouchless approach was considered sufficient and intuitive,
ment: they had to move a sphere inside five randomly
while the Leap Motion sensor was regarded as one of the
10

IEEE SyStEmS, man, & CybErnEtICS magazInE July 2 0 1 8

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2018