Skip to main content

Table 3 Pre-training results of Sundanese language models

From: Pre-trained transformer-based language models for Sundanese

Model

Training loss

Evaluation loss

Evaluation perplexity

Sundanese GPT-2

2.436

3.610

36.97

Sundanese BERT

2.860

2.845

17.20

Sundanese RoBERTa

1.965

1.952

7.04