IEEE Consumer Electronics Magazine - March 2018 - 67

FEATURE EXTRACTION
There have been many attempts to extract various features
from hands for gesture classification [35], [36]. The feature
extraction must capture the essence of a gesture. It is very
critical in hand-gesture recognition because the hand is nonrigid. Gesture variations caused by rotation, scaling, and
translation can be circumvented by using a set of features
that are invariant to these operations, such as moment
invariants. Hu moments have been considered one of the
most effective methods to describe deformable objects, and
they are widely applied in the classification of subjects [27].
Essentially, the Hu moment algorithm derives a number of
self-characteristic properties from a binary image of an
object. These properties are invariant to rotation, scale, and
translation [21].
For a digital image f ^ x, y h, with size M ) N, moment m p, q
is defined as (4), and central moment n p, q is defined as (5),
where xr and yr are the coordinates of the image center:
m p, q =

M-1 N-1

/ /

f (x, y) x p y q,

(4)

f (x, y) (x - xr ) p (y - yr ) q,

(5)

x=0 y=0
M-1 N-1

n p, q =

/ /

x=0 y=0

xr = m 10 , yr = m 01 .
m 00
m 00

(6)

The recognition of natural gestures
enables humans to control a
whole VR system without wired
equipment and acts as a free input
into the system.
NETWORK LEARNING AND TRAINING
A neural network is a powerful tool for modeling nonlinear
relationships. It has been applied to perform complex functions, including pattern recognition [31], classification [32],
identification [33], and so on. It is also able to learn and predict with time, just as a humanlike entity.
We exploit a feed-forward network with a single hidden
layer (seen in Figure 4). It is trained using backpropagation,
where Hu moments and corresponding gesture labels are
used as input and output vectors to train the network until it
approximates a function between input and output [28].
There are only three layers because of the limited number of
hand gestures to be recognized. Because there are seven Hu
moments and two gestures defined, our neural network model
has seven input neurons and two output neurons, and there
are five neurons in the hidden layer.

Then, we get normalized moments as
n p, q

h p, q =

m

p+q
+1
2
00

STEREO CALCULATION
.

(7)

Hu moments are linear combinations of the central moments,
and seven are given as follows:
h1 = h 20 + h 02,

We detect a hand in the sequent images based on color
detection and segment it out. After the hand segment, we get
the location of the hands. To reduce calculation cost, we
check the location for every m frames because movement
change is small in adjacent frames. The gap m is an experience value.
Detection is carried out on one of the image pairs caught
by binocular cameras. We analyze the gestures before their

h2 = (h 20 - h 02) 2 + 4h 112,
h3 = (h 30 - 3h 12) 2 + (3h 21 - h 03) 2,
h4 = (h 30 + h 12) 2 + (h 21 + h 03) 2,

h1

h5 = (h 30 - 3h 12) (h 30 + h 12) ((h 30 + h 12) 2
- 3 (h 21 + h 03) 2) + (3h 21 - h 03) (h 21 + h 03)
(3 (h 30 + h 12) 2 - (h 21 + h 03) 2),

h2
P0
h3

h6 = (h 20 - h 02) ((h 30 + h 12) 2 - (h 21 + h 03) 2)
+ 4h 112 (h 30 + h 12) (h 21 + h 03),

P1

...
...
h7

2

h7 = (3h 21 - h 03) (h 30 + h 12) ((h 30 + h 12)
- 3 (h 21 + h 03) 2) + (h 30 - 3h 12) (h 21 + h 03)
(3 (h 30 + h 12) 2 - (h 21 + h 03) 2) .

(8)

These features effectively describe hand gestures, and we use
them in our system for classification using neural networks.

Input Layer

Hidden Layer

Output Layer

FIGURE 4. The feed-forward network with a single hidden layer.

MARCH 2018

^

IEEE Consumer Electronics Magazine

67
Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - March 2018
