Skip to main content

Table 5 hyperparameters used in different tasks

From: Survey of transformers and towards ensemble learning using transformers for natural language processing

 

Optimizer

Learning rate

Batch size

Epochs

Sentiment analysis

Adam

1e−5

32

5

Question answering

Adam

1e−5

32

5

Name entity recognition

Adam

1e−5

64

3

ext summarization

Adam

1e−5

32

5