IEEE Systems, Man and Cybernetics Magazine - January 2022 - 36

SVMs
The results of linear and RBF SVMs are presented in
Table 4. Similar to traditional ML algorithms, the application
of SMOTE ENN resulted in a performance increase for
SVM models across the four evaluation metrics with two
exceptions: applying the SMOTE-ENN algorithm to a linear
SVM and RBF SVM resulted in a decrease in recall and
accuracy to each model, respectively.
Table 4. The SVMs' accuracy, precision,
recall, and F1 score for original and
balanced original data sets, respectively,
using the SMOTE-ENN algorithm.
F1
Algorithms
Original data set
Original data set
with SMOTE-ENN
Original data set
Original data set
with SMOTE-ENN
Accuracy Precision Recall
Linear SVM
0.9303
0.9822
0.0794
0.9949
RBF SVM
0.9399
0.876
0.2063
0.7761
1
1
0.3421
0.8739
1
0.9736
Score
0.1471
0.9841
Tree-Based Algorithms
Table 5 shows that the decision tree and AdaBoost perform
well on the unbalanced original data set compared to
other models. Similar to the majority of ML models, an
increase in performance is noticed when balancing the
data using the SMOTE-ENN algorithm. As AdaBoost is an
extension of the decision tree, a similar performance is
witnessed between it and AdaBoost, with a slight increase
in performance for AdaBoost.
MCNN
Similar to tree-based algorithms, SVMs, nearest neighbors,
and Gaussian process, MCNNs produce a high accuracy of
0.9221 on the unbalanced data set but have as significantly
lower evaluation on precision, recall, and the F1 score. As
illustrated in Table 6, MCNNs follow the trend of increased
performance on the balanced SMOTE-ENN data set.
Discussion
In this article, we explored ICP classification. We introduced
an SMOTE-ENN algorithm for balancing the data,
and its effect was examined on the classification of ICP.
We experimented on training the models with both unbalanced
and balanced data sets. The models were evaluated
using the same unbalanced testing set.
Table 5. The tree-based algorithms'
accuracy, precision, recall, and F1 score
for original and balanced original data
sets, respectively, using the SMOTE-ENN
algorithm.
Algorithms
Original data set
Original data set
with SMOTE-ENN
Original data set
Original data set
with SMOTE-ENN
Accuracy Precision Recall F1 Score
Decision tree
0.9856
0.9936
0.8889
0.9987
AdaBoost
0.9928
0.9964
0.9048
1
1
0.9936
0.95
0.9968
0.918
0.9898
0.9032
0.9942
Traditional ML Algorithms
In the nearest-neighbor model, we see an increase in performance
when balancing the data set using SMOTE ENN
because it allows for better linearity connection to the
minority class. This makes it become the nearest neighbor.
Recall shows the same trend. Precision decreases as it is
picking false positives in the expanded space due to its
near-neighbor nature.
Per the naïve Bayes model, the accuracy of the naïve
Bayes algorithm increases when applying the SMOTE-ENN
algorithm because of the likelihood term (| )Px C and
increases due to the balanced data set. An increase in likelihood
also increases the probability of picking a false positive,
which is why we see a decrease in precision from
balancing the data set using SMOTE ENN. Recall increases
slightly because it affects both the likelihood as well as the
probability of a class. The F1 score shows the same trend.
As SMOTE ENN balances the data set, it has a single
Table 6. The MCNN's accuracy, precision,
recall, and F1 score for original and
balanced original data sets, respectively,
using the SMOTE-ENN algorithm.
MCNN
Original data set
Original data set
with SMOTE-ENN
Accuracy Precision Recall F1 Score
0.9221
0.9414
0.4706
0.9164
0.4068
0.95
0.4364
0.9329
representation, which increases the covariance matrix of
single representation in the Gaussian process, causing an
increase in accuracy, precision, recall, and the F1 score.
The evaluation metrics (accuracy, precision, recall, and the
F1 score) for QDA significantly increases when data are
passed to the SMOTE-ENN algorithm because QDA is a
generalization of LDA, and it can handle the linearity introduced
by SMOTE ENN.
SVMs
Because the effect of SMOTE ENN introduces a particular
kind of linearity among the minority class, it becomes easier
36 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE January 2022
IEEE Systems, Man and Cybernetics Magazine - January 2022

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2022
