Table 10 Activation HSD test on image and face datasets
From: Evaluation of maxout activations in deep learning across several big data domains
Activation | Average accuracy (%) | Accuracy HSD | Average 100 batches time (s) | Average 100 batches training time |
|---|---|---|---|---|
ReLU 6X | 84.40 | a | 5.55 | 161.41 |
ReLU 3X | 81.68 | b | 1.84 | 58.91 |
Maxout 3-2 | 81.20 | b | 27.88 | 468.64 |
Maxout 3-1 | 81.19 | b | 13.13 | 511.67 |
ReLU 2X | 80.65 | bc | 3.78 | 161.70 |
Maxout 2-1 | 80.04 | c | 10.67 | 602.27 |
Maxout 6-1 | 80.02 | c | 2.44 | 63.68 |
Tanh | 76.82 | d | 1.73 | 136.90 |
ReLU | 76.41 | d | 1.72 | 91.39 |
SeLU | 76.33 | d | 6.92 | 407.26 |
LReLU | 76.25 | d | 3.16 | 68.90 |