From: Survey of transformers and towards ensemble learning using transformers for natural language processing
Ensemble model
Acc
XLNet+Albert+BERT+GPT2+RoBERTa
0.407
Albert+BERT+GPT2+RoBERTa
0.409
BERT+GPT2+RoBERTa
GPT2+RoBERTa
0.417
RoBERTa