Table 8 Activation HSD test on image datasets
From: Evaluation of maxout activations in deep learning across several big data domains
Activation | Average accuracy (%) | Accuracy HSD | Average 100 batches time (s) | Average 100 batches training time |
|---|---|---|---|---|
ReLU 6X | 85.19 | a | 0.25 | 13.56 |
ReLU 3X | 82.27 | b | 0.15 | 8.68 |
Maxout 6-1 | 80.68 | c | 0.19 | 9.84 |
Maxout 3-1 | 80.11 | cd | 0.12 | 6.24 |
Maxout 3-2 | 79.77 | cd | 0.19 | 8.25 |
ReLU 2X | 79.41 | cd | 0.09 | 5.12 |
Maxout 2-1 | 78.99 | d | 0.11 | 5.24 |
Tanh | 74.97 | e | 0.08 | 4.34 |
SeLU | 74.76 | e | 0.08 | 2.82 |
ReLU | 74.52 | e | 0.08 | 4.72 |
LReLU | 74.47 | e | 0.08 | 4.52 |