Table 21 Comparison table
Name of the model/method | Downstream tasks | Downstream datasets | Advantages and limitations of the model/method | Comparison with other models/methods |
|---|---|---|---|---|
BERT [3] | Sentiment analysis, question answering, NER, text summarization, topic modeling, text generation | Coronavirus tweets dataset, SQuAD 1.1, Groningen Meaning Bank corpus, CNN daily mail dataset, Disaster tweets dataset, Trump 2020 election speech dataset | Bidirectional approach for better contextual understanding. | Outperforms traditional models like LSTM and Gated Recurrent Unit (GRU). Outperforms GP2 in classification tasks. |
ALBERT [7] | Sentiment analysis, question answering, NER, topic modeling, text generation | Coronavirus tweets dataset, SQuAD 1.1, Groningen Meaning Bank corpus, Disaster tweets dataset, Trump 2020 election speech dataset | Improved efficiency and reduced parameter count. Can be trained on relatively small batch sizes. | Achieves similar results to BERT with fewer parameters and smaller batch sizes. |
RoBERTa [6] | Sentiment analysis, question answering, NER, text summarization, topic modeling, text generation | Coronavirus tweets dataset, SQuAD 1.1, Groningen Meaning Bank corpus, CNN daily mail dataset, Disaster tweets dataset, Trump 2020 election speech dataset | Training with dynamic masking for better generalization. Computationally expensive | Outperforms in many benchmarks |
XLNet [5] | Sentiment analysis, question answering, NER, topic modeling, text generation | Coronavirus tweets dataset, SQuAD 1.1, Groningen Meaning Bank corpus, Disaster tweets dataset, Trump 2020 election speech dataset | Overcomes limitations of sequential pre-training.Requires longer training times than BERT | Outperforms BERT and ALBERT in some benchmarks |
GPT2 [4] | Sentiment analysis, question answering, topic modeling, text generation | Coronavirus tweets dataset, SQuAD 1.1, Disaster tweets dataset, Trump 2020 election speech dataset | Pre-trained on large corpus of text for generalization | Achieves high performance in language generation tasks |
Our proposed ensemble learning models | Sentiment analysis, NER, text generation | Coronavirus tweets dataset, Groningen Meaning Bank corpus, Trump 2020 election speech dataset | Combines strengths of multiple models for better accuracy. Requires additional computation for inference | Outperforms individual models in sentiment analysis and NER tasks |