IEEE Systems, Man and Cybernetics Magazine - October 2020 - 47

[10]. The advantage of VAT as opposed to other visual techniques is its ability to highlight the potential number of clusters in the data by suitably reordering the objects. This is
achieved by reordering the dissimilarity matrix of the input
data using a modified Prim's algorithm and visually estimating the number of clusters that appear as the dark blocks
along the diagonal of reordered dissimilarity image (RDI).
Figure 1 shows how a VAT tool helps determine the number
of clusters from a dissimilarity matrix. The VAT tool is widely applicable to large real-world data sets, including big data.
It also allows for the display of reordered dissimilarity data,
which can be accessed from the original data, O . If O has
any missing components, then any existing data-imputation
schemes can be used to fill in the missing parts of the data
before applying a VAT.
Extensions of VAT for Handling Big Data
An Improved VAT for Improved Contrast
Suppose we have pairwise dissimilarity matrix D of a set
of n objects, a VAT generally portrays D as n # n -image
u h, where the objects are reordered to reveal hidden
I^ D

Dissimilarity Matrix

cluster structures as "dark blocks" along the diagonal of
the image [16]. However, a VAT fails to clearly display
the dark block if the data have complex structures. On the
other hand, an improved VAT (iVAT) proposes the
improved performance of a VAT by transforming the reoru using a graph-theoretic geodered dissimilarity matrix D
desic distance. Evidently, iVAT significantly enhances the
separation of the "dark blocks" in VAT images [11].
A Scalable Visual Assessment of Cluster
Tendency for Large Data Sets
Although the VAT tool finds its usefulness in many IoT
applications, it can be computationally expensive as the
size of the data set grows. An algorithm is said to be scalable if there is a linear increase in the runtime complexity
with the increase in the number of observations in the
input data [17]. A VAT has a runtime complexity of O ^ N 2h,
which is not attractive for large data sets. On the other
hand, the scalable visual assessment of cluster tendency
(sVAT) algorithm uses a sample-based version of a
VAT that can handle large data sets [18]. An sVAT chooses
sample size n from the complete set of objects,

Dissimilarity Image

0
0.62

0.62
0

0.19
0.81

0.28
0.9

0.02
0.64

0.19

0.81

0.08

0.17

0.28
0.02

0.9
0.64

0.08
0.17

0
0.26

0.26
0

VAT

~
D=

0
0.08

0.08
0

0.26
0.17

0.28
0.19

0.9
0.81

0.26

0.17

0.02

0.64

0.28
0.9

0.19
0.81

0.02
0.64

0
0.62

0.62
0

Reordered Dissimilarity Matrix

~
=I

Ordered Dissimilarity Image

Figure 1. An illustration of how a VAT reorders the dissimilarity matrix. D is the dissimilarity matrix obtained

from objects in O. From D, we note that it is difficult to determine how many clusters are present. I represents
the dissimilarity image, Du is the VAT reordered matrix obtained after applying VAT, and Iu is the reordered
u,
dissimilarity image. From the pair ^Du , Iuh, we note that there are two clusters of block sizes 4 × 4 and 1 × 1 in D
and it is evident from the visual substructure suggested by Iu.

O c tob e r 2020

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE

IEEE Systems, Man and Cybernetics Magazine - October 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2020