Pre-trained transformer-based language models for Sundanese

Journal of Big Data

Table 5 Evaluation result of Sundanese Twitter emotion classification

Model	#Params	Accuracy	F1-Macro
Baseline
Linear SVC with TF-IDF [18]	30K	96.43	96.36
Sundanese (Ours)
Sundanese GPT-2	124M	94.84	94.75
Sundanese BERT	110M	96.82	96.75
Sundanese RoBERTa	124M	98.41	98.43
Indonesian
IndoBERT [31]	124M	96.43	96.42
Multilingual
mBERT [6]	167M	96.83	96.79
XLM-RoBERTa [20]	278M	95.24	95.22