Table 2 Comparison of classic machine learning and modern transformer-based models

Data requirements

Requires large labeled dataset

Can work with a smaller labeled dataset

Feature engineering

Requires manual feature engineering

Automatically learn features from data

Model complexity

Simple models, such as logistic regression or SVM

Complex models, such as BERT, and GPT-2

Training time

Faster

Slower

Accuracy

Lower

Higher

Handling of contextual

Information

Limited

Strong ability to handle contextual information

Handling of

Unstructured data

Limited

Strong ability to handle unstructured data

Generalization ability

Can generalize well

Can generalize better