From: Evaluation of maxout activations in deep learning across several big data domains
Activation | Average accuracy (%) | Accuracy HSD | Average 100 batches time (s) | Average 100 batches training time |
---|---|---|---|---|
SeLU | 69.70 | a | 0.12 | 13.01 |
Tanh | 69.02 | ab | 0.11 | 22.62 |
Maxout 3-1 | 68.97 | ab | 0.14 | 24.77 |
Maxout 2-1 | 68.55 | ab | 0.12 | 21.50 |
ReLU 2x | 68.55 | ab | 0.11 | 24.66 |
Maxout 3-2 | 68.52 | ab | 0.19 | 30.79 |
Maxout 6-1 | 68.37 | b | 0.13 | 25.05 |
ReLU | 68.22 | b | 0.11 | 23.53 |
LReLU | 67.82 | b | 0.12 | 27.15 |