Skip to main content

Table 3 Cluster evaluation on Yahoo! Answers dataset

From: The performance of BERT as data representation of text clustering

Method

AG news

ACC

NMI

ARI

TFIDF + KM

0.3568 ± 0.0059

0.2135 ± 0.0067

0.2121 ± 0.0068

BERT + Max + I + KM

0.3018 ± 0.0131

0.1495 ± 0.0095

0.1479 ± 0.0095

BERT + Max + LN + KM

0.3285 ± 0.0140

0.1797 ± 0.0132

0.1782 ± 0.0132

BERT + Max + N + KM

0.3229 ± 0.0145

0.1739 ± 0.0137

0.1724 ± 0.0137

BERT + Max + MM + KM

0.226 ± 0.0058

0.088 ± 0.0057

0.0864 ± 0.0057

BERT + Mean + I + KM

0.357 ± 0.0079

0.2134 ± 0.0077

0.212 ± 0.0077

BERT + Mean + LN + KM

0.3741 ± 0.0057

0.2302 ± 0.0055

0.2288 ± 0.0055

BERT + Mean + N + KM

0.373 ± 0.0071

0.2286 ± 0.0066

0.2273 ± 0.0066

BERT + Mean + MM + KM

0.1718 ± 0.0066

0.0511 ± 0.0050

0.0493 ± 0.0050

TFIDF + EFCM

0.2482 ± 0.0081

0.1177 ± 0.0018

0.1161 ± 0.0018

BERT + Max + I +  EFCM

0.2484 ± 0.0070

0.122 ± 0.0029

0.1204 ± 0.0029

BERT + Max + LN +  EFCM

0.2454 ± 0.0069

0.1302 ± 0.0015

0.1287 ± 0.0015

BERT + Max + N +  EFCM

0.2374 ± 0.0051

0.1255 ± 0.0014

0.1239 ± 0.0014

BERT + Max + MM +  EFCM

0.2043 ± 0.0098

0.0706 ± 0.0044

0.0689 ± 0.0044

BERT + Mean + I +  EFCM

0.2486 ± 0.0077

0.1174 ± 0.0016

0.1158 ± 0.0016

BERT + Mean + LN +  EFCM

0.2522 ± 0.0037

0.1253 ± 0.0012

0.1237 ± 0.0012

BERT + Mean + N +  EFCM

0.2523 ± 0.0032

0.1252 ± 0.0010

0.1236 ± 0.0010

BERT + Mean + MM +  EFCM

0.1632 ± 0.0066

0.0415 ± 0.0019

0.0397 ± 0.0019

TFIDF + DEC

0.4024 ± 0.0282

0.2176 ± 0.0154

0.1621 ± 0.0202

BERT + Max + I +  DEC

0.1061 ± 0.0143

0.003 ± 0.0074

0.0015 ± 0.0038

BERT + Max + LN +  DEC

0.3969 ± 0.0186

0.2301 ± 0.0143

0.1761 ± 0.0133

BERT + Max + N +  DEC

0.1 ± 0

0 ± 0

0 ± 0

BERT + Max + MM +  DEC

0.1713 ± 0.0708

0.0539 ± 0.0638

0.0312 ± 0.0397

BERT + Mean + I +  DEC

0.4661 ± 0.0282

0.286 ± 0.0121

0.2317 ± 0.0193

BERT + Mean + LN +  DEC

0.4754 ± 0.0266

0.2907 ± 0.0119

0.2339 ± 0.0172

BERT + Mean + N +  DEC

0.427 ± 0.0292

0.2613 ± 0.013

0.1992 ± 0.0172

BERT + Mean + MM +  DEC

0.1 ± 0

0.0001 ± 0

0 ± 0

TFIDF + IDEC

0.3975 ± 0.0235

0.2243 ± 0.0109

0.1474 ± 0.0111

BERT + Max + I +  IDEC

0.1326 ± 0.0354

0.0225 ± 0.0241

0.0135 ± 0.0158

BERT + Max + LN +  IDEC

0.4058 ± 0.0182

0.2394 ± 0.0129

0.1881 ± 0.0131

BERT + Max + N +  IDEC

0.1242 ± 0.0342

0.0193 ± 0.0275

0.0097 ± 0.0144

BERT + Max + MM +  IDEC

0.1694 ± 0.0511

0.0504 ± 0.0497

0.0278 ± 0.0301

BERT + Mean + I +  IDEC

0.477 ± 0.0294

0.2988 ± 0.0126

0.2445 ± 0.0199

BERT + Mean + LN +  IDEC

0.487 ± 0.0258

0.3019 ± 0.0118

0.247 ± 0.0167

BERT + Mean + N +  IDEC

0.4308 ± 0.0303

0.2687 ± 0.0134

0.2078 ± 0.0170

BERT + Mean + MM +  IDEC

0.1015 ± 0.0029

0.0081 ± 0.005

7E-05 ± 0.0004

  1. The feature extraction and normalization strategies are abbreviated into Max for max pooling, Mean for mean pooling, I for identity normalization, LN for layer normalization, N for standard normalization, and MM for min–max normalization. The deviations denote the standard deviation of the metric from 50 repetitions. The values in bold denote the highest value in every metric in each text clustering algorithm. While the method in bold, if there are any, is the best performing method in each text clustering algorithm.