Systems, Man & Cybernetics - July 2015 - 24

memory (RAM) running MATLAB
2014a. The Gaussian-kernel ELMs
able at http://yann.lecun.com/
and SVMs required more than
exdb/mnist/) is probably the
All of the hidden
16-GB RAM, so we ran them on
most widely adopted data set
nodes in an ELM are
a high-performance cluster with
for evaluating deep architecindependent of the
2-GHz processors and 128 -GB
ture performance. It consists
RAM running MATLAB 2013a.
of two data sets with 28 # 28
training data as well
In Table 1, we report on the
images of handwritten digits,
as each other.
performance of the EDLT using
one for training (60,000 images)
the original MNIST (without any
and the other for testing (10,000
affine or elastic distortion [48]),
images).
LeafSnap, and ORL Face data
◆ The LeafSnap data set [46]
sets. MNIST and ORL Face common splits have been
(available at http://yann.lecun.com/exdb/mnist/) has
adopted. For LeafSnap, five-fold cross validation has
7,719 images of leaves taken by mobile devices in outbeen conducted on images downsampled to 32 # 32 pixdoor environments.
els and converted to grayscale.
◆ The ORL Face data set (available at http://www.cl.cam.
The results show that, for every data set, the EDLT
ac.uk/research/dtg/attarchive/facedatabase.html) has
meets the performance of the state-of-the-art architec64 # 64 images of 40 distinct subjects acquired at diftures both in terms of recognition performance and comferent times, with lighting variations and changing
putational training times. While the EDLT achieves the
facial expressions.
best recognition rates on all three data sets, the perforThe comparisons are given with respect to DBNs [16],
mance varies little between the adopted approaches. In
DBMs [17], SAEs [18], SDAEs [18], standard ELMs with
particular, it is worth noting that, for the MNIST data set,
random feature projections, Gaussian-kernel ELMs [47],
the final EDLT structure consists of 364 nodes, of which
multilayer ELMs [47], random forest of DTs [29], and suponly 29 reached a depth of 16. Since the EDLT borrows the
port vector machines (SVMs) with a radial basis function
NT structure, depth is determined during training by the
(RBF) kernel. For all the data sets, the raw pixels' intensihomogeneity condition. In our case, a partition is homoties are the input.
geneous; hence, the node becomes a leaf if all its patterns
We conducted the experiments on a desktop computer
belong to the same class. To control depth/overfitting, a
with an i7 3770 3.4-GHz core and 16-GB random access
◆ The MNIST data set [36] (avail-

Table 1. A performance comparison of an EDLT with state-of-the-art deep architectures.
*
Comparisons
have been carried out on the unmodified/unprocessed MNIST,
LeafSnap, and ORL Face data sets. Results for Gaussian-kernel ELMs and SVMs have
been computed using a faster machine. Note how, on a huge data set (like MNIST),
by exploiting graphics processing unit parallelism, the proposed approach has lower
training time than ELMs.
MNIST

Algorithm

ORL Face

EDLT

99.06 (! 0.03)

339.72

47.61 (! 1.03)

129.21

95.10 (! 1.52)

52.01

Multilayer ELM [47]

99.02 (! 0.04)

464.51

36.19 (! 0.93)

58.43

94.23 (! 0.04)

37.65

ELM random features

97.39 (! 0.1)

389.39

32.04 (! 0.87)

27.80

93.90 (! 1.08)

14.89

Gaussian-kernel ELM [47]

98.75 (! 0.09)

790.96

40.91 (! 1.02)

407.21

94.41 (! 1.21)

259.11

DBN [16]

98.87 (! 0.06)

20,580

33.12 (! 1.21)

1,566

94.79 (! 1.89)

1,066

DBM [17]

99.05 (! 0.07)

68,246

35.43 (! 1.30)

5,420

94.03 (! 1.67)

2,109

SAE [18]

98.61 (! 0.07)

9,891

22.90 (! 2.12)

931.77

81.80 (! 3.38)

480.94

SDAE [18]

98.72 (! 0.06)

13,707

24.19 (! 1.77)

1,047

82.31 (! 2.90)

600.67

Random forest (1,000 trees) [29]

98.47 (! 0.08)

17,746

46.37 (! 1.04)

1,289

90.10 (! 2.38)

810.28

SVM with RBF kernel

96.60 (! 0.09)

29,438

25.76 (! 1.52)

2,797

84.20 (! 2.97)

305.50

The best results are in boldface font.

24

LeafSnap

Accuracy (%)
Accuracy (%)
Accuracy (%)
[Standard
Training Time
[Standard
Training Time
[Standard
Training Time
Deviation (%)]
(s)
Deviation (%)]
(s)
Deviation (%)]
(s)

IEEE Systems, Man, & Cybernetics Magazine July 2015
http://yann.lecun.com/ http://yann.lecun.com/exdb/mnist/ http://www.cl.cam http://www.ac.uk/research/dtg/attarchive/facedatabase.html
Table of Contents for the Digital Edition of Systems, Man & Cybernetics - July 2015
