IEEE Systems, Man and Cybernetics Magazine - April 2020 - 54

Experimental Setup
We can hardly glorify this little illustration by labeling it as an
experiment, but we still need to explain how we made the
comparisons as fair as possible. We sampled the George data
[Figure 1 (b)] from the full set [Figure 1 (a)]. The prototypeselection methods that we used are listed in Table 1 with the
results. In addition to the techniques listed in the previous
section, we included Wilson's method followed by Hart's. This
combined approach (hybrid type) often leads to a small and
accurate reference set.
We took care that all our random methods carried out
exactly the same number of evaluations of the criterion
Table 2. The results from the experiment
with noisy George data.
Method

Type

Error Rate
(%)

Number of
Prototypes

Time
(s)

1-NN

-

16.24

1,000

0.49

Hart

C

20

415

27.87

Wilson

E

8.43

806

0.48

Wilson and
Hart

H

9.68

91

19.3

RNGE

E

8.17

794

3.97

RNN

C

21.18

355

33.94

RMHC

H

24.7

10

76.7

RMHC

H

15.65

20

77.74

RMHC

H

14.38

100

80.67

RMHC

H

15.61

200

82.14

MC1

H

21.25

10

74.18

MC1

H

15.62

20

74.91

MC1

H

15.88

100

76.5

MC1

H

14.28

200

79.76

GA

H

15.85

10

75.64

GA

H

11.08

20

77.3

GA

H

9.92

100

80.82

GA

H

12.7

197

83.03

Boldface indicates that the result is in the Pareto front in terms of error-rate/
reference-set size.

54	

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2020

function (a 1-NN error rate on the sampled George data).
The parameters in this experiment were as follows:
◆◆ MC1: number of iterations: T = 12,000
◆◆ GA: population size: K = 40, number of generations:
W = 300
◆◆ RMHC: number of chromosomes evolved (separately):
K = 40; number of mutations: W = 300.
During the second leg of the experiment, we contaminated George with label noise by flipping the labels of 10%
of the sampled data to a different class. Figure 2 shows the
classification regions of the 1-NN with the contaminated
set. George looks exploded here.
Results
Table 1 presents the results with the clean data, and
Table  2 gives the results with the noisy data. To make
more sense of the numbers, we will use a scatterplot. The
x axis is the logarithm of the number of retained prototypes out of the initial 1,000. We chose the logarithmic
scale for the sole purpose of making the graphs less
crowded at the smaller cardinalities. The y axis is the
1-NN classification error on the full data (the whole of
George). An ideal point would sit at ( ln (3) = 1.0986, 0),
where we have one prototype of each class and zero error.
The closer the point is to the origin, the better the method.
The outcomes are shown in Figure  3 for the noise-free
George and in Figure 4 for the noisy George.
In both figures, each prototype-selection method is
shown with a yellow marker. Circles represent hybrid methods, triangles indicate condensing, and squares signal editing. The thick blue line is the Pareto front; that is, the
collection of nondominated methods, which are highlighted in boldface in the respective tables. Note that we can
choose the number of prototypes for MC1, RMHC, and GA.
The versions of a method for different numbers of prototypes are shown as line graphs. In addition, next to each

0.26
0.24
0.22
1-NN Error Rate

prespecifying the number of prototypes. However, due
to the crossover, there may be repeated prototypes
within a chromosome. This means that M is an upper
limit on the number of prototypes for the GA.
The George data set and MATLAB code for this illustration are available at https://github.com/LucyKuncheva/
instance_selection.

MC1

0.2
0.18

0.16 RMHC
0.14

GA

0.12
0.1

Wilson and
Hart

0.08
0.06

RNN Hart

Wilson
RNGE

2

2.5

3 3.5 4 4.5 5 5.5 6 6.5
log (Number of Retained Prototypes)

1-NN
7

Figure 3. The scatterplot of the results for the noise-free
George data. The blue line represents the Pareto front.


https://github.com/LucyKuncheva/instance_selection https://github.com/LucyKuncheva/instance_selection

IEEE Systems, Man and Cybernetics Magazine - April 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - April 2020

Contents
IEEE Systems, Man and Cybernetics Magazine - April 2020 - Cover1
IEEE Systems, Man and Cybernetics Magazine - April 2020 - Cover2
IEEE Systems, Man and Cybernetics Magazine - April 2020 - Contents
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 2
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 3
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 4
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 5
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 6
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 7
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 8
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 9
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 10
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 11
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 12
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 13
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 14
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 15
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 16
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 17
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 18
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 19
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 20
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 21
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 22
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 23
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 24
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 25
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 26
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 27
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 28
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 29
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 30
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 31
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 32
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 33
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 34
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 35
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 36
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 37
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 38
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 39
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 40
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 41
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 42
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 43
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 44
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 45
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 46
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 47
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 48
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 49
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 50
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 51
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 52
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 53
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 54
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 55
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 56
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 57
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 58
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 59
IEEE Systems, Man and Cybernetics Magazine - April 2020 - 60
IEEE Systems, Man and Cybernetics Magazine - April 2020 - Cover3
IEEE Systems, Man and Cybernetics Magazine - April 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com