From: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Model | Main finding | Depth | Dataset | Error rate | Input size | Year |
---|---|---|---|---|---|---|
AlexNet | Utilizes Dropout and ReLU | 8 | ImageNet | 16.4 | \(227 \times 227 \times 3\) | 2012 |
NIN | New layer, called ‘mlpconv’, utilizes GAP | 3 | CIFAR-10, CIFAR-100, MNIST | 10.41, 35.68, 0.45 | \(32 \times 32 \times 3\) | 2013 |
ZfNet | Visualization idea of middle layers | 8 | ImageNet | 11.7 | \(224 \times 224 \times 3\) | 2014 |
VGG | Increased depth, small filter size | 16, 19 | ImageNet | 7.3 | \(224 \times 224 \times 3\) | 2014 |
GoogLeNet | Increased depth,block concept, different filter size, concatenation concept | 22 | ImageNet | 6.7 | \(224 \times 224 \times 3\) | 2015 |
Inception-V3 | Utilizes small filtersize, better feature representation | 48 | ImageNet | 3.5 | \(229 \times 229 \times 3\) | 2015 |
Highway | Presented the multipath concept | 19, 32 | CIFAR-10 | 7.76 | \(32 \times 32 \times 3\) | 2015 |
Inception-V4 | Divided transform and integration concepts | 70 | ImageNet | 3.08 | \(229 \times 229 \times 3\) | 2016 |
ResNet | Robust against overfitting due to symmetry mapping-based skip links | 152 | ImageNet | 3.57 | \(224 \times 224 \times 3\) | 2016 |
Inception-ResNet-v2 | Introduced the concept of residual links | 164 | ImageNet | 3.52 | \(229 \times 229 \times 3\) | 2016 |
FractalNet | Introduced the concept of Drop-Path as regularization | 40,80 | CIFAR-10 | 4.60 | \(32 \times 32 \times 3\) | 2016 |
CIFAR-100 | 18.85 | |||||
WideResNet | Decreased the depth and increased the width | 28 | CIFAR-10 | 3.89 | \(32 \times 32 \times 3\) | 2016 |
CIFAR-100 | 18.85 | |||||
Xception | A depthwise convolutionfollowed by a pointwise convolution | 71 | ImageNet | 0.055 | \(229 \times 229 \times 3\) | 2017 |
Residual attention neural network | Presented the attention technique | 452 | CIFAR-10, CIFAR-100 | 3.90, 20.4 | \(40 \times 40\times 3\) | 2017 |
Squeeze-and-excitation networks | Modeled interdependencies between channels | 152 | ImageNet | 2.25 | \(229 \times 229 \times 3\) | 2017 |
\(224 \times 224 \times 3\) | ||||||
\(320 \times 320 \times 3\) | ||||||
DenseNet | Blocks of layers; layers connected to each other | 201 | CIFAR-10, CIFAR-100,ImageNet | 3.46, 17.18, 5.54 | \(224 \times 224 \times 3\) | 2017 |
Competitive squeeze and excitation network | Both residual and identity mappings utilized to rescale the channel | 152 | CIFAR-10 | 3.58 | \(32 \times 32 \times 3\) | 2018 |
CIFAR-100 | 18.47 | |||||
MobileNet-v2 | Inverted residual structure | 53 | ImageNet | – | \(224 \times 224 \times 3\) | 2018 |
CapsuleNet | Pays attention to special relationships between features | 3 | MNIST | 0.00855 | \(28 \times 28 \times 1\) | 2018 |
HRNetV2 | High-resolution representations | – | ImageNet | 5.4 | \(224 \times 224 \times 3\) | 2020 |