Skip to main content

Table 19 The obtained results for evaluation metrics from running boosting algorithms on multi-class imbalanced big datasets

From: Boosting methods for multi-class imbalanced data classification: an experimental review

Dataset Algorithm
AdaBoost.MH SAMME CatBoost LogitBoost GradientBoost XGBoost MEBoost SMOTEBoost RUSBoost LightGBM AdaC1 AdaC2 AdaC3 AdaCost
FARS
 MAUC 0.5382 0.4754 0.5330 0.5428 0.2112 0.5108 0.5543 0.4546 0.3708 0.4755 0.5970 0.3327 0.5190 0.4306
± 0.0022 ± 0.0129 ± 0.0030 ± 0.0039 ± 0.2228 ± 0.0029 ± 0.0029 ± 0.0153 ± 0.0195 ± 0.0089 ± 0.0117 0.0138±  ± 0.0093 ± 0.130
 MMCC 0.1803 0.0907 0.1772 0.1962 − 0.3315 0.1384 0.2078 0.0580 − 0.1258 0.1084 0.2107 0.4295 0.3771 − 0.1482
± 0.0027 ± 0.0134 ± 0.0043 ± 0.0052 ± 0.0401 ± 0.0037 ± 0.0034 ± 0.0207 ± 0.0202 ± 0.0166 ± 0.0070 0.0261±  ± 0.0032 ± 0.0243
 G-mean 0.0000 0.0000 0.0000 0.0000 0.0031 0.0000 0.0000 0.0000 0.0368 0.0000 0.1013 0.0000 0.1215 0.0608
± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0072 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0422 ± 0.0000 ± 0.0700 ± 0.0000 ± 0.0579 ± 0.0159
KDD Cup’99
 MAUC 0.9251 0.9267 0.9333 0.9637 0.3057 0.8755 0.9468 0.9485 0.7874 0.4601 0.8380 0.8553 0.9440 0.6618
± 0.0158 ± 0.0109 ± 0.0155 ± 0.0112 ± 0.0218 ± 0.0078 ± 0.0036 ± 0.0172 ± 0.0294 ± 0.0274 ± 0.0169 ± 0.0250 ± 0.0042 ± 0.0468
 MMCC 0.8523 0.8320 0.8106 0.9119 0.0236 0.7120 0.8873 0.8122 0.3392 0.0171 0.3678 0.4477 0.6201 0.2050
± 0.0000 ± 0.0124 ± 0.0180 ± 0.0161 ± 0.0312 ± 0.0109 ± 0.0066 ± 0.0268 ± 0.0423 ± 0.0496 ± 0.0186 ± 0.0309 ± 0.0090 ± 0.0428
 G-mean 0.9398 0.9411 0.0000 0.9728 0.0649 0.0000 0.9535 0.8696 0.5072 0.0000 0.2637 0.3621 0.6894 0.0564
± 0.0156 ± 0.0055 ± 0.0000 ± 0.0101 ± 0.0179 ± 0.0000 ± 0.0058 ± 0.0292 ± 0.0375 ± 0.0000 ± 0.0294 ± 0.0431 ± 0.0124 ± 0.0222
Covtype
 MAUC 0.6533 0.4085 0.6624 0.8367 0.7003 0.5732 0.7722 0.4737 0.4002 0.7886 0.7416 0.4301 0.6978 0.6379
± 0.0000 ± 0.0033 ± 0.0026 0.0022 ± 0.9982 ± 0.0008 ± 00.002 ± 0.0101 ± 0.0172 ± 0.0120 ± 0.0014 ± 0.0000 ± 0.0006 ± 0.0020
 MMCC 0.2046 − 0.1357 0.2276 0.5189 0.2874 0.0791 0.3963 0.0757 − 0.1335 0.4593 0.3395 − 0.1344 0.2995 0.2146
± 0.0012 ± 0.0052 ± 00.004 ± 0.0033 ± 0.0117 ± 0.0012 ± 0.0032 ± 0.0159 ± 0.0247 ± 0.0151 ± 0.0023 ± 0.0035 ± 0.0010 ± 0.0030
 G-mean 0.6839 0.3885 0.6967 0.8594 0.7003 0.6038 00.796 0.7237 0.3313 0.8138 0.5551 0.0000 0.5033 0.0666
± 0.0018 ± 0.0156 ± 0.0033 ± 0.0024 ± 0.0082 ± 0.0020 ± 0.0013 ± 0.0089 ± 0.0132 ± 0.0154 ± 0.0027 ± 0.0000 ± 0.0009 ± 0.0544
Poker
 MAUC 0.0524 0.1557 0.2376 0.1765 0.1168 0.1350 0.2585 0.1659 0.1117 0.2593 0.2217 0.09357 0.2087 0.06224
± 0.0014 ± 0.0073 ± 0.0042 ± 0.0005 ± 0.0116 ± 0.0023 ± 0.0201 ± 0.0018 ± 0.0045 ± 0.0157 ± 0.0041 ± 0.0136 ± 0.0162 ± 0.0216
 MMCC − 0.4545 − 0.3696 − 0.3011 − 0.3889 − 0.3556 − 0.5599 0.2879−  − 0.2821 − 0.3352 0.275–0 − 0.3790 − 0.8569 − 0.2284 − 0.4425
± 0.0035 ± 0.0140 ± 0.0038 ± 0.0126 ± 0.0083 ± 0.0082 ± 0.0162 ± 0.0076 ± 0.0086 ± 0.0241 ± 0.0089 ± 0.0272 ± 0.0098 ± 0.0155
 G-mean 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0018 0.0000
± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0000 ± 0.0040 ± 0.0000
Average over MMCC 0.1957 0.1044 0.2286 0.3095 − 0.0940 0.0924 0.3009 0.1660 − 0.0638 0.0775 0.1348 − 0.0285 0.2671 − 0.0428
  1. The best performance is shown in italic for each dataset