IEEE Systems, Man, and Cybernetics Magazine - January 2018 - 49

valuable addition to your clustering
arsenal. Bezdek then introduces the
functions used to develop the clustering algorithms. Their minima, ideally,
will result in a good set of clusters. In
practice, the book shows it does not
always happen.
A discussion of cluster validity algorithms follows. Because there are
choices of parameters and starting
points for three of the algorithms discussed here (and you can use different
distance metrics for the fourth), the
question of how good a set of clusters
or data partitions arises. The book includes a very useful discussion of this
problem for a general set of clusters.
Chapter 5 examines the problem
of getting good features to accurately
describe your objects so that you can
effectively cluster them. It also covers
the problem of deciding which features to use when you are given a set
(without labels). It discusses principal
component analysis for reducing the
number of features, if there are many.
Real-world examples on relatively simple data are given. Projections of data
are discussed, as are relational data issues. The important problems of data
normalization are brought up so that a
feature with a wide range of values (say,
zero to one million) does not obscure a
useful feature in the range (zero to one).
Chapter 6 analyzes the widely
used k-means, hard c-means, or hard
k-means (c / k). It also covers FCM,
which has a bigger search space (i.e.,
it is more general than k-means; I leave
it to the reader to find out if FCM is a
generalization of k-means). You will
be able to compare and contrast the
algorithms and learn that k-means is
not strictly solved by alternating optimization. There is a discussion about
how to match a final clustering partition to ground truth (data labeled in
some way).
Throughout the book, multiple
ways to compare clusters are given,
which is a strong point. Experimental evidence is typically an average
of more than z trials with different
initializations, and z = 100 is common.
There is stability to the given examples
as well as an important discussion of

model parameters (e.g., c or k, the disthor shows that, when using minimum
tance metric) for the algorithms, their
spanning trees in an algorithm for
effect, and how to choose them. For
single linkage, there is a relationship
instance, you can change the cluster
with graph theory. A number of examshape with the distance metric, and
ples are given that illustrate relational
you are given five good choices for
clustering's strong and weak points.
distance. The execution parameters,
How to choose the number of clusters
such as when to stop and how to start/
is critical for these algorithms, as for
initialize, are discussed in depth. There
n objects, they start with n clusters
is a very helpful exposition on how to
and end with one. There are reliable
choose the number
methods to choose
of clusters via validity
the number of clusThis book includes
measures. It is noted
ters given.
that there are other
In Chapter 9, the
necessary mathways to optimize the
properties of the four
ematical details
clustering algorithms.
general approaches
as well as pointers
A discussion is given
are given in depth.
to any required
for k-means of an apComparisons beproach that will protween the approachmathematics
vide a lower bound
es on the same data
required to fully
on the objective funcsets show when
comprehend the
tion being minimized.
choosing one of them
subtleties of
Chapter 7 examwill produce clusters
ines EM clustering,
that are much more
an algorithm.
which is a probabililike what humans
ty-based method. As
would do (so we are
with FCM, at the end of the process,
looking at low-dimensional data). In the
you will need to assign an object to
process of comparing approaches, the
a cluster by maximum membership
adjusted Rand index and other ways to
(FCM) or maximum probability (EM).
compare partitions are investigated.
The general assumption in EM is that
Chapter 10 discusses alternating
you have a mixture of Gaussians; you
optimization. This chapter is useful if
get (hyper-) ellipsoidal clusters. You
you want a refresher on AO or if you
search for probability density funcare looking to deeply understand it.
tions, and the algorithm is alternatOtherwise, it can be skipped if you
ing optimization (AO). The chapter
are willing to believe it works and
provides examples as well as experiwill (generally) converge, though you
mental comparisons with FCM and
might want to know about trap points
k-means on the well-known iris data.
and local extrema.
It also provides a good discussion of
Chapter 11 focuses on big data.
choosing model parameters. There is
Clustering algorithms, including the
an enlightening exposition of cluster
four types this book focuses on, may
validity indexes for EM and a good
not scale well when there are 14 milcomparison on actual data.
lion 4,096 -dimensional examples
Chapter 8 on relational clustering
(say, we try to cluster ImageNet exnotes that there are relational veramples from the outputs of the last
sions of some of the previously studlayer of a convolutional neural netied algorithms, but it focuses on the
work). Bezdek comes up with seven
venerable linkage algorithms. It is
Vs to describe big data. The chapter
argued that single and complete linkcontains a discussion of a number
age generally performs well. Average
of algorithms for clustering big data.
linkage is shown to have some issues
You will see they connect with the
that leave the author to mostly drop it.
earlier algorithms (mostly), but some
There is an evaluation of when single
are pretty different and a few have
linkage will work and complete linkno connection. This chapter really
age will fail and vice versa. The aumoves beyond the primer concept in a
Ja nua r y 2 01 8

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE

Table of Contents for the Digital Edition of IEEE Systems, Man, and Cybernetics Magazine - January 2018